Loading ...
Sorry, an error occurred while loading the content.

Feedback on implementing client side HTTP caching

Expand Messages
  • Sidu Ponnappa
    Hello everyone, One of my colleagues has been working on implementing client side HTTP caching on a open source project of ours. He s had a hard time finding
    Message 1 of 22 , Jan 25, 2011
    • 0 Attachment
      Hello everyone,

      One of my colleagues has been working on implementing client side HTTP
      caching on a open source project of ours.  He's had a hard time
      finding comprehensive documentation, so he put together
      https://github.com/kaiwren/wrest/blob/caching/Caching.markdown by
      working through the HTTP RFC as well as Firefox source.

      I'd appreciate it if those of you that know how this should work
      (especially in the context of REST) could take a look and let us know
      if there is anything that we've missed. A large portion of that
      document has already been implemented; we expect to release these
      features in about a week as a part of Wrest 1.1.

      Thanks,
      Sidu.
      http://c42.in
    • Erlend Hamnaberg
      For inspiration check out my Open Source Java HTTP caching library: http://httpcache4j.codehaus.org the source is also available here:
      Message 2 of 22 , Jan 25, 2011
      • 0 Attachment
        For inspiration check out my Open Source Java HTTP caching library:
        http://httpcache4j.codehaus.org

        the source is also available here:
        https://github.com/hamnis/httpcache4j

        --

        Erlend

        On Tue, Jan 25, 2011 at 10:06 AM, Sidu Ponnappa <lorddaemon@...> wrote:
         

        Hello everyone,

        One of my colleagues has been working on implementing client side HTTP
        caching on a open source project of ours.  He's had a hard time
        finding comprehensive documentation, so he put together
        https://github.com/kaiwren/wrest/blob/caching/Caching.markdown by
        working through the HTTP RFC as well as Firefox source.

        I'd appreciate it if those of you that know how this should work
        (especially in the context of REST) could take a look and let us know
        if there is anything that we've missed. A large portion of that
        document has already been implemented; we expect to release these
        features in about a week as a part of Wrest 1.1.

        Thanks,
        Sidu.
        http://c42.in


      • Moore, Jonathan
        Also, my colleagues and I implemented a client-side cache for Apache HttpComponents HttpClient, which was just included in the 4.1 GA release of HttpClient:
        Message 3 of 22 , Jan 25, 2011
        • 0 Attachment
          Also, my colleagues and I implemented a client-side cache for Apache HttpComponents HttpClient, which was just included in the 4.1 GA release of HttpClient:

          http://www.apache.org/dist/httpcomponents/httpclient/RELEASE_NOTES.txt

          Jon

          From: rest-discuss@yahoogroups.com [rest-discuss@yahoogroups.com] on behalf of Erlend Hamnaberg [ngarthl@...]
          Sent: Tuesday, January 25, 2011 6:50 AM
          To: Sidu Ponnappa
          Cc: rest-discuss@yahoogroups.com
          Subject: Re: [rest-discuss] Feedback on implementing client side HTTP caching

           

          For inspiration check out my Open Source Java HTTP caching library:
          http://httpcache4j.codehaus.org

          the source is also available here:
          https://github.com/hamnis/httpcache4j

          --

          Erlend

          On Tue, Jan 25, 2011 at 10:06 AM, Sidu Ponnappa <lorddaemon@...> wrote:
           

          Hello everyone,

          One of my colleagues has been working on implementing client side HTTP
          caching on a open source project of ours.  He's had a hard time
          finding comprehensive documentation, so he put together
          https://github.com/kaiwren/wrest/blob/caching/Caching.markdown by
          working through the HTTP RFC as well as Firefox source.

          I'd appreciate it if those of you that know how this should work
          (especially in the context of REST) could take a look and let us know
          if there is anything that we've missed. A large portion of that
          document has already been implemented; we expect to release these
          features in about a week as a part of Wrest 1.1.

          Thanks,
          Sidu.
          http://c42.in


        • Subbu Allamaraju
          Any reason why a forward proxy cache would not work? For instance you could set up squid or traffic server as a forward proxy. Subbu
          Message 4 of 22 , Jan 25, 2011
          • 0 Attachment
            Any reason why a forward proxy cache would not work? For instance you could set up squid or traffic server as a forward proxy.

            Subbu

            On Jan 25, 2011, at 5:56 AM, Moore, Jonathan wrote:

            >
            >
            > Also, my colleagues and I implemented a client-side cache for Apache HttpComponents HttpClient, which was just included in the 4.1 GA release of HttpClient:
            >
            > http://www.apache.org/dist/httpcomponents/httpclient/RELEASE_NOTES.txt
            >
            > Jon
            > From: rest-discuss@yahoogroups.com [rest-discuss@yahoogroups.com] on behalf of Erlend Hamnaberg [ngarthl@...]
            > Sent: Tuesday, January 25, 2011 6:50 AM
            > To: Sidu Ponnappa
            > Cc: rest-discuss@yahoogroups.com
            > Subject: Re: [rest-discuss] Feedback on implementing client side HTTP caching
            >
            >
            > For inspiration check out my Open Source Java HTTP caching library:
            > http://httpcache4j.codehaus.org
            >
            > the source is also available here:
            > https://github.com/hamnis/httpcache4j
            >
            > --
            >
            > Erlend
            >
            >
            > On Tue, Jan 25, 2011 at 10:06 AM, Sidu Ponnappa <lorddaemon@...> wrote:
            >
            > Hello everyone,
            >
            > One of my colleagues has been working on implementing client side HTTP
            > caching on a open source project of ours. He's had a hard time
            > finding comprehensive documentation, so he put together
            > https://github.com/kaiwren/wrest/blob/caching/Caching.markdown by
            > working through the HTTP RFC as well as Firefox source.
            >
            > I'd appreciate it if those of you that know how this should work
            > (especially in the context of REST) could take a look and let us know
            > if there is anything that we've missed. A large portion of that
            > document has already been implemented; we expect to release these
            > features in about a week as a part of Wrest 1.1.
            >
            > Thanks,
            > Sidu.
            > http://c42.in
            >
            >
            >
            >
            >
            >
          • Subbu Allamaraju
            httpcache4j illustrates the wrong way to do HTTP caching. Clients should not have to say things like HTTPCache cache = new HTTPCache( new MemoryCacheStorage(),
            Message 5 of 22 , Jan 25, 2011
            • 0 Attachment
              httpcache4j illustrates the wrong way to do HTTP caching. Clients should not have to say things like

              HTTPCache cache = new HTTPCache(
              new MemoryCacheStorage(),
              HTTPClientResponseResolver.createMultithreadedInstance()
              );
              response = cache.doCachedRequest(request);

              in their code. They would make regular HTTP requests (including appropriate headers), and a caching proxy can provide a cached response if available or forward the request to upstream servers if no cached response is found. In other words, the client would simply do

              response = request.doGet(); // pseudo code

              The point of the uniform interface is to let some intermediary along the network do the job.

              Subb

              On Jan 25, 2011, at 3:50 AM, Erlend Hamnaberg wrote:

              >
              >
              > For inspiration check out my Open Source Java HTTP caching library:
              > http://httpcache4j.codehaus.org
              >
              > the source is also available here:
              > https://github.com/hamnis/httpcache4j
              >
              > --
              >
              > Erlend
              >
              > On Tue, Jan 25, 2011 at 10:06 AM, Sidu Ponnappa <lorddaemon@...> wrote:
              >
              > Hello everyone,
              >
              > One of my colleagues has been working on implementing client side HTTP
              > caching on a open source project of ours. He's had a hard time
              > finding comprehensive documentation, so he put together
              > https://github.com/kaiwren/wrest/blob/caching/Caching.markdown by
              > working through the HTTP RFC as well as Firefox source.
              >
              > I'd appreciate it if those of you that know how this should work
              > (especially in the context of REST) could take a look and let us know
              > if there is anything that we've missed. A large portion of that
              > document has already been implemented; we expect to release these
              > features in about a week as a part of Wrest 1.1.
              >
              > Thanks,
              > Sidu.
              > http://c42.in
              >
              >
              >
              >
              >
            • David Stanek
              ... Why not cache on the client side? In mobile applications where the network calls are expensive it makes sense to cache locally. -- David blog:
              Message 6 of 22 , Jan 25, 2011
              • 0 Attachment
                On Tue, Jan 25, 2011 at 8:56 PM, Subbu Allamaraju <subbu@...> wrote:
                 

                httpcache4j illustrates the wrong way to do HTTP caching. Clients should not have to say things like

                HTTPCache cache = new HTTPCache(
                new MemoryCacheStorage(),
                HTTPClientResponseResolver.createMultithreadedInstance()
                );
                response = cache.doCachedRequest(request);

                in their code. They would make regular HTTP requests (including appropriate headers), and a caching proxy can provide a cached response if available or forward the request to upstream servers if no cached response is found. In other words, the client would simply do

                response = request.doGet(); // pseudo code

                The point of the uniform interface is to let some intermediary along the network do the job.


                Why not cache on the client side? In mobile applications where the network calls are expensive it makes sense to cache locally. 


                --
                David
                blog: http://www.traceback.org
                twitter: http://twitter.com/dstanek
              • Nathan
                ... Just want to check I m reading you correctly, that client side caching should not be implemented, and one should instead lean on intermediaries along the
                Message 7 of 22 , Jan 25, 2011
                • 0 Attachment
                  Subbu Allamaraju wrote:
                  > httpcache4j illustrates the wrong way to do HTTP caching. Clients should not have to say things like
                  >
                  > HTTPCache cache = new HTTPCache(
                  > new MemoryCacheStorage(),
                  > HTTPClientResponseResolver.createMultithreadedInstance()
                  > );
                  > response = cache.doCachedRequest(request);
                  >
                  > in their code. They would make regular HTTP requests (including appropriate headers), and a caching proxy can provide a cached response if available or forward the request to upstream servers if no cached response is found. In other words, the client would simply do
                  >
                  > response = request.doGet(); // pseudo code
                  >
                  > The point of the uniform interface is to let some intermediary along the network do the job.

                  Just want to check I'm reading you correctly, that client side caching
                  should not be implemented, and one should instead lean on
                  intermediaries along the network?
                • Subbu Allamaraju
                  ... It is the latter. There are of course exceptions where the client - as an agent like a browser - does it own caching. On a broader note, getting caching
                  Message 8 of 22 , Jan 25, 2011
                  • 0 Attachment
                    On Jan 25, 2011, at 6:40 PM, Nathan wrote:

                    > Subbu Allamaraju wrote:
                    >> httpcache4j illustrates the wrong way to do HTTP caching. Clients should not have to say things like
                    >> HTTPCache cache = new HTTPCache(
                    >> new MemoryCacheStorage(),
                    >> HTTPClientResponseResolver.createMultithreadedInstance()
                    >> );
                    >> response = cache.doCachedRequest(request);
                    >> in their code. They would make regular HTTP requests (including appropriate headers), and a caching proxy can provide a cached response if available or forward the request to upstream servers if no cached response is found. In other words, the client would simply do
                    >> response = request.doGet(); // pseudo code
                    >> The point of the uniform interface is to let some intermediary along the network do the job.
                    >
                    > Just want to check I'm reading you correctly, that client side caching should not be implemented, and one should instead lean on intermediaries along the network?

                    It is the latter. There are of course exceptions where the client - as an agent like a browser - does it own caching.

                    On a broader note, getting caching right at scale is a non-trivial problem, and it is better to let a proxy cache deal with it.

                    Subbu
                  • Subbu Allamaraju
                    Sure, a browser is another example. There is nothing to forbid a client from doing its own caching, but in general, it is better to delegate this to a proxy
                    Message 9 of 22 , Jan 25, 2011
                    • 0 Attachment
                      Sure, a browser is another example. There is nothing to forbid a client from doing its own caching, but in general, it is better to delegate this to a proxy cache..

                      On Jan 25, 2011, at 6:03 PM, David Stanek wrote:

                      > On Tue, Jan 25, 2011 at 8:56 PM, Subbu Allamaraju <subbu@...> wrote:
                      >
                      > httpcache4j illustrates the wrong way to do HTTP caching. Clients should not have to say things like
                      >
                      > HTTPCache cache = new HTTPCache(
                      > new MemoryCacheStorage(),
                      > HTTPClientResponseResolver.createMultithreadedInstance()
                      > );
                      > response = cache.doCachedRequest(request);
                      >
                      > in their code. They would make regular HTTP requests (including appropriate headers), and a caching proxy can provide a cached response if available or forward the request to upstream servers if no cached response is found. In other words, the client would simply do
                      >
                      > response = request.doGet(); // pseudo code
                      >
                      > The point of the uniform interface is to let some intermediary along the network do the job.
                      >
                      >
                      > Why not cache on the client side? In mobile applications where the network calls are expensive it makes sense to cache locally.
                      >
                      >
                      > --
                      > David
                      > blog: http://www.traceback.org
                      > twitter: http://twitter.com/dstanek
                    • Erlend Hamnaberg
                      Unsurprisingly I respectfully disagree that this is the wrong way. A client cache becomes just another intermediary along the way in the uniform interface. Why
                      Message 10 of 22 , Jan 25, 2011
                      • 0 Attachment
                        Unsurprisingly I respectfully disagree that this is the wrong way.

                        A client cache becomes just another intermediary along the way in the uniform interface.
                        Why should a non-trivial REST application be treated differently than a browser?

                        In an application that serves some hundred or more requests per minute, a browser/client cache shows significant less network traffic.
                        It assumes of course that the client cache works the same as any other intermediary, which httpcache4j's case does.

                        --
                        Erlend

                        On Wed, Jan 26, 2011 at 8:01 AM, Subbu Allamaraju <subbu@...> wrote:
                        Sure, a browser is another example. There is nothing to forbid a client from doing its own caching, but in general, it is better to delegate this to a proxy cache..

                        On Jan 25, 2011, at 6:03 PM, David Stanek wrote:

                        > On Tue, Jan 25, 2011 at 8:56 PM, Subbu Allamaraju <subbu@...> wrote:
                        >
                        > httpcache4j illustrates the wrong way to do HTTP caching. Clients should not have to say things like
                        >
                        > HTTPCache cache = new HTTPCache(
                        > new MemoryCacheStorage(),
                        > HTTPClientResponseResolver.createMultithreadedInstance()
                        > );
                        > response = cache.doCachedRequest(request);
                        >
                        > in their code. They would make regular HTTP requests (including appropriate headers), and a caching proxy can provide a cached response if available or forward the request to upstream servers if no cached response is found. In other words, the client would simply do
                        >
                        > response = request.doGet(); // pseudo code
                        >
                        > The point of the uniform interface is to let some intermediary along the network do the job.
                        >
                        >
                        > Why not cache on the client side? In mobile applications where the network calls are expensive it makes sense to cache locally.
                        >
                        >
                        > --
                        > David
                        > blog: http://www.traceback.org
                        > twitter: http://twitter.com/dstanek


                      • Sidu Ponnappa
                        Thank you, Nathan. I ll look into the new draft RFC. Erlend and Jonthan, thanks for the links to your respective projects - we ll take a look at those too. I
                        Message 11 of 22 , Jan 26, 2011
                        • 0 Attachment
                          Thank you, Nathan. I'll look into the new draft RFC.

                          Erlend and Jonthan, thanks for the links to your respective projects - we'll take a look at those too. I haven't looked yet, but is there a suite of functional tests that exists that we can use to determine compliance? We're building one as we go, but if there's something that already exists that is comprehensive, it would be of great help.

                          Thanks,
                          Sidu.
                          http://c42.in

                          On Tue, Jan 25, 2011 at 5:20 PM, Erlend Hamnaberg <ngarthl@...> wrote:
                          For inspiration check out my Open Source Java HTTP caching library:
                          http://httpcache4j.codehaus.org

                          the source is also available here:
                          https://github.com/hamnis/httpcache4j

                          --

                          Erlend

                          On Tue, Jan 25, 2011 at 10:06 AM, Sidu Ponnappa <lorddaemon@...> wrote:
                           

                          Hello everyone,

                          One of my colleagues has been working on implementing client side HTTP
                          caching on a open source project of ours.  He's had a hard time
                          finding comprehensive documentation, so he put together
                          https://github.com/kaiwren/wrest/blob/caching/Caching.markdown by
                          working through the HTTP RFC as well as Firefox source.

                          I'd appreciate it if those of you that know how this should work
                          (especially in the context of REST) could take a look and let us know
                          if there is anything that we've missed. A large portion of that
                          document has already been implemented; we expect to release these
                          features in about a week as a part of Wrest 1.1.

                          Thanks,
                          Sidu.
                          http://c42.in



                        • Erik Mogensen
                          ... HTTPCache4j is most useful in a client, but we ve since used it in server-to-server communications too. We generally use HTTPCache4j to do all of our HTTP
                          Message 12 of 22 , Jan 26, 2011
                          • 0 Attachment
                            On Wed, Jan 26, 2011 at 8:01 AM, Subbu Allamaraju <subbu@...> wrote:
                             

                            Sure, a browser is another example. There is nothing to forbid a client from doing its own caching, but in general, it is better to delegate this to a proxy cache..



                            HTTPCache4j is most useful in a client, but we've since used it in server-to-server communications too.  We generally use HTTPCache4j to do all of our HTTP communication, using it more as a HTTP client rather than a specific HTTP cache; the cache is just there.  And as long as the server doesn't provide any caching directives, HTTPCache4j doesn't provide any value.

                            The real value is reaped when we (e.g. in production) need to add caching.  Adding a real intermediary is not always feasible, and it's nice to know that all of our clients honour caching directives correctly.  All we need to do to increase performance is to drop in a header or two in the server's responses, and just like that, client perceived performance increases many orders of magnitude.

                            The only gripe I have with HTTPCache4j is the names of methods and so on.  "doCachedRequest" ought to be called simply "request"...  ;-)
                            -- 
                            -mogsie-
                          • Mike Kelly
                            I think the crux here is that a caching component can/should be modeled as an intermediary, even if it is client-side i.e. implemented as an intermediary
                            Message 13 of 22 , Jan 26, 2011
                            • 0 Attachment
                              I think the crux here is that a caching component can/should be modeled as an intermediary, even if it is client-side i.e. implemented as an intermediary handling requests locally before they hit the wire.

                              There should be no need for the requesting code to be concerned with caching because it's unnecessary - you don't make 'caching' XHR requests because normal XHR requests can receive locally cached responses from the browser cache, which is effectively acting as an intermediary. This is exactly what layering and caching are for.

                              Saying that, I imagine using an 'off-the-shelf' forward _shared_ proxy locally may present some problems in terms of handling responses with the private directive. Are there options for squid and/or traffic to turn on a 'private' mode?

                              Cheers,
                              Mike


                              On Wed, Jan 26, 2011 at 7:01 AM, Subbu Allamaraju <subbu@...> wrote:
                              Sure, a browser is another example. There is nothing to forbid a client from doing its own caching, but in general, it is better to delegate this to a proxy cache..

                              On Jan 25, 2011, at 6:03 PM, David Stanek wrote:

                              > On Tue, Jan 25, 2011 at 8:56 PM, Subbu Allamaraju <subbu@...> wrote:
                              >
                              > httpcache4j illustrates the wrong way to do HTTP caching. Clients should not have to say things like
                              >
                              > HTTPCache cache = new HTTPCache(
                              > new MemoryCacheStorage(),
                              > HTTPClientResponseResolver.createMultithreadedInstance()
                              > );
                              > response = cache.doCachedRequest(request);
                              >
                              > in their code. They would make regular HTTP requests (including appropriate headers), and a caching proxy can provide a cached response if available or forward the request to upstream servers if no cached response is found. In other words, the client would simply do
                              >
                              > response = request.doGet(); // pseudo code
                              >
                              > The point of the uniform interface is to let some intermediary along the network do the job.
                              >
                              >
                              > Why not cache on the client side? In mobile applications where the network calls are expensive it makes sense to cache locally.
                              >
                              >
                              > --
                              > David
                              > blog: http://www.traceback.org
                              > twitter: http://twitter.com/dstanek



                              ------------------------------------

                              Yahoo! Groups Links

                              <*> To visit your group on the web, go to:
                                 http://groups.yahoo.com/group/rest-discuss/

                              <*> Your email settings:
                                 Individual Email | Traditional

                              <*> To change settings online go to:
                                 http://groups.yahoo.com/group/rest-discuss/join
                                 (Yahoo! ID required)

                              <*> To change settings via email:
                                 rest-discuss-digest@yahoogroups.com
                                 rest-discuss-fullfeatured@yahoogroups.com

                              <*> To unsubscribe from this group, send an email to:
                                 rest-discuss-unsubscribe@yahoogroups.com

                              <*> Your use of Yahoo! Groups is subject to:
                                 http://docs.yahoo.com/info/terms/


                            • Moore, Jonathan
                              Hi, yes, the caching module has a suite of unit tests to test compliance with the HTTP/1.1 spec. These are separated out into requirements (MUST/MUST NOT) and
                              Message 14 of 22 , Jan 26, 2011
                              • 0 Attachment
                                Hi, yes, the caching module has a suite of unit tests to test compliance with the HTTP/1.1 spec. These are separated out into requirements (MUST/MUST NOT) and recommendations (SHOULD/SHOULD NOT). Here are links to them in the source repository:


                                And


                                The caching module is implemented as a decorator around a regular HttpClient — meaning you instantiate the cache when you instantiate the client but can then "ignore" caching when you use it. I.e. You do:

                                HttpClient regularBackend = new DefaultHttpClient(…);
                                HttpClient theClientToUse = new CachingHttpClient(regularBackend, …);

                                Jon
                                ........
                                Jon Moore
                                Comcast Interactive Media



                                From: Sidu Ponnappa <lorddaemon@...>
                                Date: Wed, 26 Jan 2011 14:01:03 +0530
                                To: <rest-discuss@yahoogroups.com>
                                Subject: Re: [rest-discuss] Feedback on implementing client side HTTP caching

                                 

                                Thank you, Nathan. I'll look into the new draft RFC.

                                Erlend and Jonthan, thanks for the links to your respective projects - we'll take a look at those too. I haven't looked yet, but is there a suite of functional tests that exists that we can use to determine compliance? We're building one as we go, but if there's something that already exists that is comprehensive, it would be of great help.

                                Thanks,
                                Sidu.
                                http://c42.in

                                On Tue, Jan 25, 2011 at 5:20 PM, Erlend Hamnaberg <ngarthl@...> wrote:

                                For inspiration check out my Open Source Java HTTP caching library:
                                http://httpcache4j.codehaus.org

                                the source is also available here:
                                https://github.com/hamnis/httpcache4j

                                --

                                Erlend

                                On Tue, Jan 25, 2011 at 10:06 AM, Sidu Ponnappa <lorddaemon@...> wrote:
                                 

                                Hello everyone,

                                One of my colleagues has been working on implementing client side HTTP
                                caching on a open source project of ours.  He's had a hard time
                                finding comprehensive documentation, so he put together
                                https://github.com/kaiwren/wrest/blob/caching/Caching.markdown by
                                working through the HTTP RFC as well as Firefox source.

                                I'd appreciate it if those of you that know how this should work
                                (especially in the context of REST) could take a look and let us know
                                if there is anything that we've missed. A large portion of that
                                document has already been implemented; we expect to release these
                                features in about a week as a part of Wrest 1.1.

                                Thanks,
                                Sidu.
                                http://c42.in



                              • Moore, Jonathan
                                I think there s clearly a place for client-side caching — the moral equivalent of a browser cache, but for a general application. Even if all the calls are
                                Message 15 of 22 , Jan 26, 2011
                                • 0 Attachment
                                  I think there's clearly a place for client-side caching — the moral equivalent of a browser cache, but for a general application. Even if all the calls are intra-datacenter, I might be better off having some cache entries stored in my application's address space for performance reasons. This can be coupled with a local forward proxy cache to implement layered caching (much as we have L1/L2 caches in CPUs).

                                  I think Subbu's point here wasn't that client-side caching was problematic, but that the part of the program issuing the requests ought not to have to behave differently in terms of programming interface due to the presence or absence of a client-side cache. The client-side cache ought to be something I can "drop in" or "wire in" without making real code changes — this takes advantage of the fact that HTTP caching is largely meant to be semantically transparent.

                                  Jon
                                  ........
                                  Jon Moore
                                  Comcast Interactive Media



                                  From: David Stanek <dstanek@...>
                                  Date: Tue, 25 Jan 2011 21:03:16 -0500
                                  To: Subbu Allamaraju <subbu@...>
                                  Cc: Erlend Hamnaberg <ngarthl@...>, Sidu Ponnappa <lorddaemon@...>, <rest-discuss@yahoogroups.com>
                                  Subject: Re: [rest-discuss] Feedback on implementing client side HTTP caching

                                   

                                  On Tue, Jan 25, 2011 at 8:56 PM, Subbu Allamaraju <subbu@...> wrote:
                                   

                                  httpcache4j illustrates the wrong way to do HTTP caching. Clients should not have to say things like

                                  HTTPCache cache = new HTTPCache(
                                  new MemoryCacheStorage(),
                                  HTTPClientResponseResolver.createMultithreadedInstance()
                                  );
                                  response = cache.doCachedRequest(request);

                                  in their code. They would make regular HTTP requests (including appropriate headers), and a caching proxy can provide a cached response if available or forward the request to upstream servers if no cached response is found. In other words, the client would simply do

                                  response = request.doGet(); // pseudo code

                                  The point of the uniform interface is to let some intermediary along the network do the job.


                                  Why not cache on the client side? In mobile applications where the network calls are expensive it makes sense to cache locally. 


                                  --
                                  David
                                  blog: http://www.traceback.org
                                  twitter: http://twitter.com/dstanek

                                • mike amundsen
                                  FWIW, adding client-side caching support is relatively simple on the Windows platform. Windows exposes base-level client-side caching services to the .NET
                                  Message 16 of 22 , Jan 26, 2011
                                  • 0 Attachment
                                    FWIW, adding client-side caching support is relatively simple on the
                                    Windows platform. Windows exposes base-level client-side caching
                                    services to the .NET runtime. Enabling it for any HTTP calls takes one
                                    line of code (on the initialized HttpWebRequest object):

                                    request.CachePolicy = new HttpRequestCachePolicy(HttpRequestCacheLevel.Default);

                                    This was pointed out rather nicely in REST In Practice[1].

                                    In my view, HTTP Client applications SHOULD understand and honor
                                    caching directives. It MAY be possible to off-load this work to a
                                    proxy. Whether that proxy is internal or external will be a function
                                    of the programming environment, framework, etc available to the
                                    developer.


                                    [1] http://my.safaribooksonline.com/book/web-development/web-services/9781449383312/scaling-out/implementing_caching_in_net

                                    mca
                                    http://amundsen.com/blog/
                                    http://twitter.com@mamund
                                    http://mamund.com/foaf.rdf#me


                                    #RESTFest 2010
                                    http://rest-fest.googlecode.com




                                    On Wed, Jan 26, 2011 at 02:01, Subbu Allamaraju <subbu@...> wrote:
                                    > Sure, a browser is another example. There is nothing to forbid a client from doing its own caching, but in general, it is better to delegate this to a proxy cache..
                                    >
                                    > On Jan 25, 2011, at 6:03 PM, David Stanek wrote:
                                    >
                                    >> On Tue, Jan 25, 2011 at 8:56 PM, Subbu Allamaraju <subbu@...> wrote:
                                    >>
                                    >> httpcache4j illustrates the wrong way to do HTTP caching. Clients should not have to say things like
                                    >>
                                    >> HTTPCache cache = new HTTPCache(
                                    >> new MemoryCacheStorage(),
                                    >> HTTPClientResponseResolver.createMultithreadedInstance()
                                    >> );
                                    >> response = cache.doCachedRequest(request);
                                    >>
                                    >> in their code. They would make regular HTTP requests (including appropriate headers), and a caching proxy can provide a cached response if available or forward the request to upstream servers if no cached response is found. In other words, the client would simply do
                                    >>
                                    >> response = request.doGet(); // pseudo code
                                    >>
                                    >> The point of the uniform interface is to let some intermediary along the network do the job.
                                    >>
                                    >>
                                    >> Why not cache on the client side? In mobile applications where the network calls are expensive it makes sense to cache locally.
                                    >>
                                    >>
                                    >> --
                                    >> David
                                    >> blog: http://www.traceback.org
                                    >> twitter: http://twitter.com/dstanek
                                    >
                                    >
                                    >
                                    > ------------------------------------
                                    >
                                    > Yahoo! Groups Links
                                    >
                                    >
                                    >
                                    >
                                  • Nathan
                                    ... In that case, sorry, but I have to disagree. The concept of client+cache is pretty fundamental to the REST style, and mentioned repeatedly throughout the
                                    Message 17 of 22 , Jan 26, 2011
                                    • 0 Attachment
                                      Subbu Allamaraju wrote:
                                      > On Jan 25, 2011, at 6:40 PM, Nathan wrote:
                                      >
                                      >> Subbu Allamaraju wrote:
                                      >>> httpcache4j illustrates the wrong way to do HTTP caching. Clients should not have to say things like
                                      >>> HTTPCache cache = new HTTPCache(
                                      >>> new MemoryCacheStorage(),
                                      >>> HTTPClientResponseResolver.createMultithreadedInstance()
                                      >>> );
                                      >>> response = cache.doCachedRequest(request);
                                      >>> in their code. They would make regular HTTP requests (including appropriate headers), and a caching proxy can provide a cached response if available or forward the request to upstream servers if no cached response is found. In other words, the client would simply do
                                      >>> response = request.doGet(); // pseudo code
                                      >>> The point of the uniform interface is to let some intermediary along the network do the job.
                                      >> Just want to check I'm reading you correctly, that client side caching should not be implemented, and one should instead lean on intermediaries along the network?
                                      >
                                      > It is the latter. There are of course exceptions where the client - as an agent like a browser - does it own caching.

                                      In that case, sorry, but I have to disagree.

                                      The concept of client+cache is pretty fundamental to the REST style,
                                      and mentioned repeatedly throughout the dissertation, even a tertiary
                                      glance at the text and figures should make this more than clear.
                                      Similarly the HTTP specification(s) spend a significant amount of time
                                      discussing client / private caching, and catering for it (see
                                      p4-conditional and p6-cache).

                                      Hence why agents like the browsers, do their own caching.

                                      It's very good practise to cache at the edges, every edge, including
                                      client and server, and of course at intermediaries along the way; I
                                      personally could not bring myself to suggest to anybody that creating
                                      a client+cache is a bad idea, quite the opposite in fact, it's a
                                      wonderful idea.

                                      That said, it is of course wise to take the design principals of the
                                      REST style and apply them to the design of your components, both
                                      client and cache connectors should be hidden by an agent, exposing a
                                      nice interface to communicate - as Mike Kelly noted earlier with
                                      regards to XHR.

                                      I have to say that I can't find anything wrong with httpcache4j, it's
                                      a good example of a client cache, but one would be wise to wrap it up
                                      in another layer / interface which makes the cache and client
                                      connectors invisible to "the end user", but then httpcache4j is one of
                                      those connectors that you take, and wrap up inside your own agent,
                                      even if that agent is only intended to expose a nice programmable
                                      interface for application developers to use (again, like XHR).

                                      Best,

                                      Nathan
                                    • Subbu Allamaraju
                                      ... Fair enough :) When considering operability and scalability, or when the client is running on a web server (like a front-end talking to some other service)
                                      Message 18 of 22 , Jan 26, 2011
                                      • 0 Attachment
                                        On Jan 26, 2011, at 8:30 AM, Nathan wrote:

                                        > In that case, sorry, but I have to disagree.

                                        Fair enough :)

                                        When considering operability and scalability, or when the client is running on a web server (like a front-end talking to some other service) I would not opt for client side (in particular in process) caching.

                                        Subbu
                                      • Subbu Allamaraju
                                        ... I don t think either of them cache private responses by default. There may be plugins or mods to support that. Subbu
                                        Message 19 of 22 , Jan 26, 2011
                                        • 0 Attachment
                                          On Jan 26, 2011, at 1:38 AM, Mike Kelly wrote:

                                          > Saying that, I imagine using an 'off-the-shelf' forward _shared_ proxy locally may present some problems in terms of handling responses with the private directive. Are there options for squid and/or traffic to turn on a 'private' mode?

                                          I don't think either of them cache private responses by default. There may be plugins or mods to support that.

                                          Subbu
                                        • Nathan
                                          ... Depends very much on the use case for me, if I m just using HTTP for RPC usage (interacting with some realtime service like a payment service, leaning
                                          Message 20 of 22 , Jan 26, 2011
                                          • 0 Attachment
                                            Subbu Allamaraju wrote:
                                            > On Jan 26, 2011, at 8:30 AM, Nathan wrote:
                                            >
                                            >> In that case, sorry, but I have to disagree.
                                            >
                                            > Fair enough :)
                                            >
                                            > When considering operability and scalability, or when the client is running on a web server (like a front-end talking to some other service) I would not opt for client side (in particular in process) caching.

                                            Depends very much on the use case for me, if I'm just using HTTP for
                                            RPC usage (interacting with some realtime service like a payment
                                            service, leaning heavily on POST) then there's little point (if
                                            there's nothing to cache!) - if however, you're interacting with the
                                            web at large (anything that involves GETting stuff from URIs) then
                                            caching will definitely be beneficial.

                                            Quite possibly this mail chain, like many REST related ones, is also
                                            suffering from massive overuse of the term 'client', using it to refer
                                            to an HTTP Client, a Client+Cache, an Agent, a Component, an
                                            Application and so forth. That could have some bearing on the messages
                                            being conveyed.

                                            Best,

                                            Nathan
                                          • Subbu Allamaraju
                                            ... Minor correction - I meant to say in-process and not client-side . ... Yes - client side proxy caching is fairly common.
                                            Message 21 of 22 , Jan 26, 2011
                                            • 0 Attachment
                                              On Jan 26, 2011, at 8:55 AM, Nathan wrote:

                                              > Subbu Allamaraju wrote:
                                              >> On Jan 26, 2011, at 8:30 AM, Nathan wrote:
                                              >>> In that case, sorry, but I have to disagree.
                                              >> Fair enough :)
                                              >> When considering operability and scalability, or when the client is running on a web server (like a front-end talking to some other service) I would not opt for client side (in particular in process) caching.

                                              Minor correction - I meant to say "in-process" and not "client-side".

                                              >
                                              > Depends very much on the use case for me, if I'm just using HTTP for RPC usage (interacting with some realtime service like a payment service, leaning heavily on POST) then there's little point (if there's nothing to cache!) - if however, you're interacting with the web at large (anything that involves GETting stuff from URIs) then caching will definitely be beneficial.

                                              Yes - client side proxy caching is fairly common.

                                              >
                                              > Quite possibly this mail chain, like many REST related ones, is also suffering from massive overuse of the term 'client', using it to refer to an HTTP Client, a Client+Cache, an Agent, a Component, an Application and so forth. That could have some bearing on the messages being conveyed.
                                              >
                                              > Best,
                                              >
                                              > Nathan
                                            • Sidu Ponnappa
                                              Hello everyone, We got the release with caching support out of the door on Friday last week. You can install the gem using `(sudo) gem in wrest` - if you have
                                              Message 22 of 22 , Jan 31, 2011
                                              • 0 Attachment
                                                Hello everyone,

                                                We got the release with caching support out of the door on Friday last week. You can install the gem using `(sudo) gem in wrest` - if you have the time, please do try it out; we'd appreciate any feedback you may have.

                                                Links:
                                                Source: https://github.com/kaiwren/wrest
                                                Docs: http://rdoc.info/github/kaiwren/wrest/master/frames#Caching
                                                CI server: http://ci.c42.in

                                                Thanks in advance for your time.

                                                Best,
                                                Sidu.
                                                http://c42.in
                                                http://about.me/ponnappa
                                              Your message has been successfully submitted and would be delivered to recipients shortly.