Loading ...
Sorry, an error occurred while loading the content.

RE: [rest-discuss] Clients & Expiration based caching

Expand Messages
  • Moore, Jonathan (CIM)
    The Apache HttpComponents project includes a client-side cache with multiple backing stores (memory, ehcache, memcached) as of version 4.1. See
    Message 1 of 11 , Jul 1, 2011
    • 0 Attachment
      The Apache HttpComponents project includes a client-side cache with multiple backing stores (memory, ehcache, memcached) as of version 4.1.


      As one of the authors of the caching module, I agree with the assertion that getting this right is non-trivial (I think we have close to 1000 unit tests to line up requirements from RFC2616 with the implementation), so I would weigh the software investment against the operational overhead of running/scaling/maintaining the forward proxy cache.

      Jon

      From: rest-discuss@yahoogroups.com [rest-discuss@yahoogroups.com] on behalf of bryan_w_taylor [bryan_w_taylor@...]
      Sent: Friday, July 01, 2011 2:10 AM
      To: rest-discuss@yahoogroups.com
      Subject: [rest-discuss] Clients & Expiration based caching

       

      In RESTful web services cookbook recipe 9.3, Subbu recommends against having clients endpoints implement caching based on expiration headers. Instead he recommends using a forward proxy. His reasoning appears to be that correct implementation of caching is non-trivial and that getting reuse of a client-side proxy is easier, safer, and less risky.

      I'm wondering if others agree with his views. Are there any client side libraries that handle the trickiness for you? Is "it's hard to get right" a good enough reason.

    • mike amundsen
      There was a thread on this here back in January[1]: http://tech.dir.groups.yahoo.com/group/rest-discuss/message/17219 As mentioned in that thread, for the
      Message 2 of 11 , Jul 1, 2011
      • 0 Attachment
        There was a thread on this here back in January[1]:

        As mentioned in that thread, for the Windows platform, a single line of code adds very good client-side caching support[2]:
        request.CachePolicy = new HttpRequestCachePolicy(HttpRequestCacheLevel.Default);

        Options for other platforms are mentioned in that thread, too.


        mca
        http://amundsen.com/blog/
        http://twitter.com@mamund
        http://mamund.com/foaf.rdf#me


        #RESTFest 2011 - Aug 18-20
        http://restfest.org


        On Fri, Jul 1, 2011 at 09:38, Moore, Jonathan (CIM) <jonathan_moore@...> wrote:


        The Apache HttpComponents project includes a client-side cache with multiple backing stores (memory, ehcache, memcached) as of version 4.1.


        As one of the authors of the caching module, I agree with the assertion that getting this right is non-trivial (I think we have close to 1000 unit tests to line up requirements from RFC2616 with the implementation), so I would weigh the software investment against the operational overhead of running/scaling/maintaining the forward proxy cache.

        Jon

        From: rest-discuss@yahoogroups.com [rest-discuss@yahoogroups.com] on behalf of bryan_w_taylor [bryan_w_taylor@...]
        Sent: Friday, July 01, 2011 2:10 AM
        To: rest-discuss@yahoogroups.com
        Subject: [rest-discuss] Clients & Expiration based caching

         

        In RESTful web services cookbook recipe 9.3, Subbu recommends against having clients endpoints implement caching based on expiration headers. Instead he recommends using a forward proxy. His reasoning appears to be that correct implementation of caching is non-trivial and that getting reuse of a client-side proxy is easier, safer, and less risky.

        I'm wondering if others agree with his views. Are there any client side libraries that handle the trickiness for you? Is "it's hard to get right" a good enough reason.




      • Peter Williams
        On Fri, Jul 1, 2011 at 12:10 AM, bryan_w_taylor ... If you are in Ruby the resourceful gem[1] provides in-process caching. If you are working with
        Message 3 of 11 , Jul 1, 2011
        • 0 Attachment
          On Fri, Jul 1, 2011 at 12:10 AM, bryan_w_taylor
          <bryan_w_taylor@...> wrote:
          > I'm wondering if others agree with his views. Are there any client side libraries that
          > handle the trickiness for you? Is "it's hard to get right" a good enough reason.

          If you are in Ruby the resourceful gem[1] provides in-process caching.

          If you are working with public/non-sensitive data i think a caching
          proxy is an excellent choice. If, however, you are using HTTPS the
          caching proxy approach basically falls apart completely (afaik).

          [1]: http://github.com/paul/resourceful

          Peter
        • Subbu Allamaraju
          ... Not necessarily the case. If you setup the forward proxy as the one starting TLS, then the proxy can cache. The same goes for the server side too where a
          Message 4 of 11 , Jul 3, 2011
          • 0 Attachment
            On Jul 1, 2011, at 12:27 PM, Peter Williams wrote:

            > you are using HTTPS the
            > caching proxy approach basically falls apart completely (afaik).

            Not necessarily the case. If you setup the forward proxy as the one starting TLS, then the proxy can cache. The same goes for the server side too where a reverse proxy cache can terminate TLS and serve cached representations.

            Subbu
          • Mark Baker
            Python s httplib2 has decent client side caching support IME.
            Message 5 of 11 , Jul 3, 2011
            • 0 Attachment

              Python's httplib2 has decent client side caching support IME.

            • Peter Williams
              ... Are there approaches to client side tls terminating reverse proxies that do not require the proxy to rewrite all the URIs in the response representations?
              Message 6 of 11 , Jul 3, 2011
              • 0 Attachment
                On Sun, Jul 3, 2011 at 1:47 PM, Subbu Allamaraju <subbu@...> wrote:
                >> you are using HTTPS the
                >> caching proxy approach basically falls apart completely (afaik).
                >
                > Not necessarily the case. If you setup the forward proxy as the one
                > starting TLS, then the proxy can cache. The same goes for the server
                > side too where a reverse proxy cache can terminate TLS and serve
                > cached representations.

                Are there approaches to client side tls terminating reverse proxies
                that do not require the proxy to rewrite all the URIs in the response
                representations? Such rewriting might limit proxying's usefulness to
                systems based on media types with very wide reach.

                Peter
              • Subbu Allamaraju
                ... Not to my knowledge. Links in representations (non HTML cases) are still rare in the wild. Subbu
                Message 7 of 11 , Jul 5, 2011
                • 0 Attachment
                  On Jul 3, 2011, at 7:35 PM, Peter Williams wrote:

                  > On Sun, Jul 3, 2011 at 1:47 PM, Subbu Allamaraju <subbu@...> wrote:
                  >>> you are using HTTPS the
                  >>> caching proxy approach basically falls apart completely (afaik).
                  >>
                  >> Not necessarily the case. If you setup the forward proxy as the one
                  >> starting TLS, then the proxy can cache. The same goes for the server
                  >> side too where a reverse proxy cache can terminate TLS and serve
                  >> cached representations.
                  >
                  > Are there approaches to client side tls terminating reverse proxies
                  > that do not require the proxy to rewrite all the URIs in the response
                  > representations?

                  Not to my knowledge. Links in representations (non HTML cases) are still rare in the wild.

                  Subbu
                • Peter Williams
                  ... In my world they are very common. I see them in RDF, JSON and XML all the time. Peter
                  Message 8 of 11 , Jul 6, 2011
                  • 0 Attachment
                    On Tue, Jul 5, 2011 at 11:18 PM, Subbu Allamaraju <subbu@...> wrote:
                    > Not to my knowledge. Links in representations (non HTML cases) are still rare in the wild.

                    In my world they are very common. I see them in RDF, JSON and XML all the time.

                    Peter
                  • Subbu Allamaraju
                    ... Quite possible. There is a quite a bit of gap in what hypertext driven apps expect and what off-the-shelf proxies/caches can do today. In one of my
                    Message 9 of 11 , Jul 7, 2011
                    • 0 Attachment
                      On Jul 6, 2011, at 9:37 AM, Peter Williams wrote:

                      > On Tue, Jul 5, 2011 at 11:18 PM, Subbu Allamaraju <subbu@...> wrote:
                      >> Not to my knowledge. Links in representations (non HTML cases) are still rare in the wild.
                      >
                      > In my world they are very common. I see them in RDF, JSON and XML all the time.

                      Quite possible. There is a quite a bit of gap in what hypertext driven apps expect and what off-the-shelf proxies/caches can do today.

                      In one of my previous projects we did build a rewriting forward proxy to forward requests over TLS to origin servers, but the use case is not related to caching. But it can certainly be done with custom-built proxies or using proxy-specific plugin APIs.

                      Subbu
                    Your message has been successfully submitted and would be delivered to recipients shortly.