Loading ...
Sorry, an error occurred while loading the content.

Clients & Expiration based caching

Expand Messages
  • bryan_w_taylor
    In RESTful web services cookbook recipe 9.3, Subbu recommends against having clients endpoints implement caching based on expiration headers. Instead he
    Message 1 of 11 , Jun 30, 2011
    • 0 Attachment
      In RESTful web services cookbook recipe 9.3, Subbu recommends against having clients endpoints implement caching based on expiration headers. Instead he recommends using a forward proxy. His reasoning appears to be that correct implementation of caching is non-trivial and that getting reuse of a client-side proxy is easier, safer, and less risky.

      I'm wondering if others agree with his views. Are there any client side libraries that handle the trickiness for you? Is "it's hard to get right" a good enough reason.
    • Moore, Jonathan (CIM)
      The Apache HttpComponents project includes a client-side cache with multiple backing stores (memory, ehcache, memcached) as of version 4.1. See
      Message 2 of 11 , Jul 1, 2011
      • 0 Attachment
        The Apache HttpComponents project includes a client-side cache with multiple backing stores (memory, ehcache, memcached) as of version 4.1.


        As one of the authors of the caching module, I agree with the assertion that getting this right is non-trivial (I think we have close to 1000 unit tests to line up requirements from RFC2616 with the implementation), so I would weigh the software investment against the operational overhead of running/scaling/maintaining the forward proxy cache.

        Jon

        From: rest-discuss@yahoogroups.com [rest-discuss@yahoogroups.com] on behalf of bryan_w_taylor [bryan_w_taylor@...]
        Sent: Friday, July 01, 2011 2:10 AM
        To: rest-discuss@yahoogroups.com
        Subject: [rest-discuss] Clients & Expiration based caching

         

        In RESTful web services cookbook recipe 9.3, Subbu recommends against having clients endpoints implement caching based on expiration headers. Instead he recommends using a forward proxy. His reasoning appears to be that correct implementation of caching is non-trivial and that getting reuse of a client-side proxy is easier, safer, and less risky.

        I'm wondering if others agree with his views. Are there any client side libraries that handle the trickiness for you? Is "it's hard to get right" a good enough reason.

      • mike amundsen
        There was a thread on this here back in January[1]: http://tech.dir.groups.yahoo.com/group/rest-discuss/message/17219 As mentioned in that thread, for the
        Message 3 of 11 , Jul 1, 2011
        • 0 Attachment
          There was a thread on this here back in January[1]:

          As mentioned in that thread, for the Windows platform, a single line of code adds very good client-side caching support[2]:
          request.CachePolicy = new HttpRequestCachePolicy(HttpRequestCacheLevel.Default);

          Options for other platforms are mentioned in that thread, too.


          mca
          http://amundsen.com/blog/
          http://twitter.com@mamund
          http://mamund.com/foaf.rdf#me


          #RESTFest 2011 - Aug 18-20
          http://restfest.org


          On Fri, Jul 1, 2011 at 09:38, Moore, Jonathan (CIM) <jonathan_moore@...> wrote:


          The Apache HttpComponents project includes a client-side cache with multiple backing stores (memory, ehcache, memcached) as of version 4.1.


          As one of the authors of the caching module, I agree with the assertion that getting this right is non-trivial (I think we have close to 1000 unit tests to line up requirements from RFC2616 with the implementation), so I would weigh the software investment against the operational overhead of running/scaling/maintaining the forward proxy cache.

          Jon

          From: rest-discuss@yahoogroups.com [rest-discuss@yahoogroups.com] on behalf of bryan_w_taylor [bryan_w_taylor@...]
          Sent: Friday, July 01, 2011 2:10 AM
          To: rest-discuss@yahoogroups.com
          Subject: [rest-discuss] Clients & Expiration based caching

           

          In RESTful web services cookbook recipe 9.3, Subbu recommends against having clients endpoints implement caching based on expiration headers. Instead he recommends using a forward proxy. His reasoning appears to be that correct implementation of caching is non-trivial and that getting reuse of a client-side proxy is easier, safer, and less risky.

          I'm wondering if others agree with his views. Are there any client side libraries that handle the trickiness for you? Is "it's hard to get right" a good enough reason.




        • Peter Williams
          On Fri, Jul 1, 2011 at 12:10 AM, bryan_w_taylor ... If you are in Ruby the resourceful gem[1] provides in-process caching. If you are working with
          Message 4 of 11 , Jul 1, 2011
          • 0 Attachment
            On Fri, Jul 1, 2011 at 12:10 AM, bryan_w_taylor
            <bryan_w_taylor@...> wrote:
            > I'm wondering if others agree with his views. Are there any client side libraries that
            > handle the trickiness for you? Is "it's hard to get right" a good enough reason.

            If you are in Ruby the resourceful gem[1] provides in-process caching.

            If you are working with public/non-sensitive data i think a caching
            proxy is an excellent choice. If, however, you are using HTTPS the
            caching proxy approach basically falls apart completely (afaik).

            [1]: http://github.com/paul/resourceful

            Peter
          • Subbu Allamaraju
            ... Not necessarily the case. If you setup the forward proxy as the one starting TLS, then the proxy can cache. The same goes for the server side too where a
            Message 5 of 11 , Jul 3, 2011
            • 0 Attachment
              On Jul 1, 2011, at 12:27 PM, Peter Williams wrote:

              > you are using HTTPS the
              > caching proxy approach basically falls apart completely (afaik).

              Not necessarily the case. If you setup the forward proxy as the one starting TLS, then the proxy can cache. The same goes for the server side too where a reverse proxy cache can terminate TLS and serve cached representations.

              Subbu
            • Mark Baker
              Python s httplib2 has decent client side caching support IME.
              Message 6 of 11 , Jul 3, 2011
              • 0 Attachment

                Python's httplib2 has decent client side caching support IME.

              • Peter Williams
                ... Are there approaches to client side tls terminating reverse proxies that do not require the proxy to rewrite all the URIs in the response representations?
                Message 7 of 11 , Jul 3, 2011
                • 0 Attachment
                  On Sun, Jul 3, 2011 at 1:47 PM, Subbu Allamaraju <subbu@...> wrote:
                  >> you are using HTTPS the
                  >> caching proxy approach basically falls apart completely (afaik).
                  >
                  > Not necessarily the case. If you setup the forward proxy as the one
                  > starting TLS, then the proxy can cache. The same goes for the server
                  > side too where a reverse proxy cache can terminate TLS and serve
                  > cached representations.

                  Are there approaches to client side tls terminating reverse proxies
                  that do not require the proxy to rewrite all the URIs in the response
                  representations? Such rewriting might limit proxying's usefulness to
                  systems based on media types with very wide reach.

                  Peter
                • Subbu Allamaraju
                  ... Not to my knowledge. Links in representations (non HTML cases) are still rare in the wild. Subbu
                  Message 8 of 11 , Jul 5, 2011
                  • 0 Attachment
                    On Jul 3, 2011, at 7:35 PM, Peter Williams wrote:

                    > On Sun, Jul 3, 2011 at 1:47 PM, Subbu Allamaraju <subbu@...> wrote:
                    >>> you are using HTTPS the
                    >>> caching proxy approach basically falls apart completely (afaik).
                    >>
                    >> Not necessarily the case. If you setup the forward proxy as the one
                    >> starting TLS, then the proxy can cache. The same goes for the server
                    >> side too where a reverse proxy cache can terminate TLS and serve
                    >> cached representations.
                    >
                    > Are there approaches to client side tls terminating reverse proxies
                    > that do not require the proxy to rewrite all the URIs in the response
                    > representations?

                    Not to my knowledge. Links in representations (non HTML cases) are still rare in the wild.

                    Subbu
                  • Peter Williams
                    ... In my world they are very common. I see them in RDF, JSON and XML all the time. Peter
                    Message 9 of 11 , Jul 6, 2011
                    • 0 Attachment
                      On Tue, Jul 5, 2011 at 11:18 PM, Subbu Allamaraju <subbu@...> wrote:
                      > Not to my knowledge. Links in representations (non HTML cases) are still rare in the wild.

                      In my world they are very common. I see them in RDF, JSON and XML all the time.

                      Peter
                    • Subbu Allamaraju
                      ... Quite possible. There is a quite a bit of gap in what hypertext driven apps expect and what off-the-shelf proxies/caches can do today. In one of my
                      Message 10 of 11 , Jul 7, 2011
                      • 0 Attachment
                        On Jul 6, 2011, at 9:37 AM, Peter Williams wrote:

                        > On Tue, Jul 5, 2011 at 11:18 PM, Subbu Allamaraju <subbu@...> wrote:
                        >> Not to my knowledge. Links in representations (non HTML cases) are still rare in the wild.
                        >
                        > In my world they are very common. I see them in RDF, JSON and XML all the time.

                        Quite possible. There is a quite a bit of gap in what hypertext driven apps expect and what off-the-shelf proxies/caches can do today.

                        In one of my previous projects we did build a rewriting forward proxy to forward requests over TLS to origin servers, but the use case is not related to caching. But it can certainly be done with custom-built proxies or using proxy-specific plugin APIs.

                        Subbu
                      Your message has been successfully submitted and would be delivered to recipients shortly.