Loading ...
Sorry, an error occurred while loading the content.

Re: [rest-discuss] Atom and prev links

Expand Messages
  • Greg Young
    Actually we have a pretty simple reason for this caching. If we were to hide the prev that uri would have to be non-cachable. If we put it there (we are
    Message 1 of 23 , Jun 28, 2013
    View Source
    • 0 Attachment
      Actually we have a pretty simple reason for this caching.

      If we were to hide the prev that uri would have to be non-cachable. If we put it there (we are forward only) then every valid uri we have is infinitely cachable. By predefining the next only our non-existant uri is non-cachable.

      Though keeping the last existing one as non-cachable could also solve this and maybe its a better overall way.

      Cheers,

      Greg

      On Fri, Jun 28, 2013 at 1:47 PM, Markus Lanthaler <markus.lanthaler@...> wrote:
       

      On Friday, June 28, 2013 12:16 PM, Greg Young wrote:
      > The issue revolves around prev links. At some point on an atom feed
      > you will hit the current item, prev will point to the next item that
      > does not yet exist. Most implementations I can find 404 this uri and
      > the client has the logic to keep polling the 404. This seems off to
      > me, I would expect this to be a 204. Then if I change my uri structure
      > it becomes a 404, otherwise it would seem that if I change my uri
      > structure all prev uris would need to redirect to the new ones (which
      > would be a rather large undertaking).

      Why don't you just hide the next/prev link till the next/prev resource
      really exists? If really necessary, the client can keep polling the last
      resource it encountered till the next/prev link becomes available. This also
      means that you can decide how to construct the URL of the new element as
      soon as it is created instead of relying on, e.g., an autoincrement.

      Did I miss something?

      --
      Markus Lanthaler
      @markuslanthaler




      --
      Le doute n'est pas une condition agréable, mais la certitude est absurde.
    • Markus Lanthaler
      ... Well, a 404 can be cached as well, so it s not that much of a difference. Just use an Etag or something similar for the last page and drop it as soon as
      Message 2 of 23 , Jun 28, 2013
      View Source
      • 0 Attachment
        On Friday, June 28, 2013 12:58 PM, Greg Young wrote:
        > Actually we have a pretty simple reason for this caching.
        >
        > If we were to hide the prev that uri would have to be non-cachable.
        > If we put it there (we are forward only) then every valid uri we
        > have is infinitely cachable. By predefining the next only our
        > non-existant uri is non-cachable.

        Well, a 404 can be cached as well, so it's not that much of a difference.
        Just use an Etag or something similar for the last page and drop it as soon
        as you put the link in.

        If I would implement a client, I wouldn't keep polling the 404. I would just
        assume your feed is broken.


        --
        Markus Lanthaler
        @markuslanthaler
      • Greg Young
        And if I returned a 204? On Fri, Jun 28, 2013 at 3:04 PM, Markus Lanthaler ... -- Le doute n est pas une condition agréable, mais la certitude est absurde.
        Message 3 of 23 , Jun 28, 2013
        View Source
        • 0 Attachment
          And if I returned a 204?

          On Fri, Jun 28, 2013 at 3:04 PM, Markus Lanthaler <markus.lanthaler@...> wrote:
           

          On Friday, June 28, 2013 12:58 PM, Greg Young wrote:
          > Actually we have a pretty simple reason for this caching.
          >
          > If we were to hide the prev that uri would have to be non-cachable.
          > If we put it there (we are forward only) then every valid uri we
          > have is infinitely cachable. By predefining the next only our
          > non-existant uri is non-cachable.

          Well, a 404 can be cached as well, so it's not that much of a difference.
          Just use an Etag or something similar for the last page and drop it as soon
          as you put the link in.

          If I would implement a client, I wouldn't keep polling the 404. I would just
          assume your feed is broken.

          --
          Markus Lanthaler
          @markuslanthaler




          --
          Le doute n'est pas une condition agréable, mais la certitude est absurde.
        • Markus Lanthaler
          ... No, the status code that would make most sense IMO is 202.. but I would still suggest to just hide the link till the resource gets created. -- Markus
          Message 4 of 23 , Jun 28, 2013
          View Source
          • 0 Attachment
            On Friday, June 28, 2013 2:10 PM, Greg Young wrote:
            > On Fri, Jun 28, 2013 at 3:04 PM, Markus Lanthaler wrote:
            >> If I would implement a client, I wouldn't keep polling the 404.
            >> I would just assume your feed is broken.
            >
            >And if I returned a 204?

            No, the status code that would make "most" sense IMO is 202.. but I would
            still suggest to just hide the link till the resource gets created.


            --
            Markus Lanthaler
            @markuslanthaler
          • Greg Young
            btw in doing some research it would appear that RFC 5005 allows for either in the areas where reading of feeds is actually discussed. This process should be
            Message 5 of 23 , Jun 28, 2013
            View Source
            • 0 Attachment
              btw in doing some research it would appear that RFC 5005 allows for either in the areas where reading of feeds is actually discussed.
               
              This process
                 should be repeated recursively until the client encounters a prev-
                 archive link relation that has been processed (the end of the archive
                 is indicated by a missing prev-archive link relation) or an error is
                 encountered.

              prev/next reading is not discussed but would seem logical to work in the same way. I am unsure why it would be "broken" as you say to return an error. Can link me to the relevant specification?

              Thanks,

              Greg

              On Fri, Jun 28, 2013 at 4:00 PM, Markus Lanthaler <markus.lanthaler@...> wrote:
               

              On Friday, June 28, 2013 2:10 PM, Greg Young wrote:


              > On Fri, Jun 28, 2013 at 3:04 PM, Markus Lanthaler wrote:
              >> If I would implement a client, I wouldn't keep polling the 404.
              >> I would just assume your feed is broken.
              >
              >And if I returned a 204?

              No, the status code that would make "most" sense IMO is 202.. but I would
              still suggest to just hide the link till the resource gets created.

              --
              Markus Lanthaler
              @markuslanthaler




              --
              Le doute n'est pas une condition agréable, mais la certitude est absurde.
            • Greg Young
              Looks like our answer is to rename our links archive-prev/next as we intend them to be completely readable anyways. Thanks! Greg ... -- Le doute n est pas une
              Message 6 of 23 , Jun 28, 2013
              View Source
              • 0 Attachment
                Looks like our answer is to rename our links archive-prev/next as we intend them to be completely readable anyways.

                Thanks!

                Greg

                On Fri, Jun 28, 2013 at 4:19 PM, Greg Young <gregoryyoung1@...> wrote:
                btw in doing some research it would appear that RFC 5005 allows for either in the areas where reading of feeds is actually discussed.
                 
                This process
                   should be repeated recursively until the client encounters a prev-
                   archive link relation that has been processed (the end of the archive
                   is indicated by a missing prev-archive link relation) or an error is
                   encountered.

                prev/next reading is not discussed but would seem logical to work in the same way. I am unsure why it would be "broken" as you say to return an error. Can link me to the relevant specification?

                Thanks,

                Greg

                On Fri, Jun 28, 2013 at 4:00 PM, Markus Lanthaler <markus.lanthaler@...> wrote:
                 

                On Friday, June 28, 2013 2:10 PM, Greg Young wrote:


                > On Fri, Jun 28, 2013 at 3:04 PM, Markus Lanthaler wrote:
                >> If I would implement a client, I wouldn't keep polling the 404.
                >> I would just assume your feed is broken.
                >
                >And if I returned a 204?

                No, the status code that would make "most" sense IMO is 202.. but I would
                still suggest to just hide the link till the resource gets created.

                --
                Markus Lanthaler
                @markuslanthaler




                --
                Le doute n'est pas une condition agréable, mais la certitude est absurde.



                --
                Le doute n'est pas une condition agréable, mais la certitude est absurde.
              • Markus Lanthaler
                On Friday, June 28, 2013 3:19 PM, Greg Young wrote ... I didn t say that a spec prohibits it, how could it? You have to expect broken links everywhere. I just
                Message 7 of 23 , Jun 28, 2013
                View Source
                • 0 Attachment
                  On Friday, June 28, 2013 3:19 PM, Greg Young wrote
                  > btw in doing some research it would appear that RFC 5005 allows
                  > for either in the areas where reading of feeds is actually discussed.

                  > This process
                  > should be repeated recursively until the client encounters a prev-
                  > archive link relation that has been processed (the end of the archive
                  > is indicated by a missing prev-archive link relation) or an error is
                  > encountered.
                  >
                  > prev/next reading is not discussed but would seem logical to work in the
                  > same way. I am unsure why it would be "broken" as you say to return an
                  > error. Can link me to the relevant specification?

                  I didn't say that a spec prohibits it, how could it? You have to expect
                  broken links everywhere. I just said I would assume that your implementation
                  is broken because it creates links that lead to a 404.

                  Errors shouldn't control the *normal* control flow, they should be
                  exceptions.



                  --
                  Markus Lanthaler
                  @markuslanthaler
                • Greg Young
                  And doing it your way requires every uri we create to be put out twice (once as mutable and uncachable once as immutable and infinitely cachable). Take
                  Message 8 of 23 , Jun 28, 2013
                  View Source
                  • 0 Attachment
                    And doing it your way requires every uri we create to be put out twice (once as mutable and uncachable once as immutable and infinitely cachable).

                    Take archives. My current archive is therefor mutable. I have to rewrite the archive every time I add a new one. This sounds like a fairly complicated protocol as opposed to pre-writing what the next uri will be and giving some sort of status when it is not yet in existence. This becomes even more so if my archives are say monthly and widely distributed. This is especially true for us as our data is already immutable (prev/next were actually the wrong choice, we should be using archive)

                    Your client just has to say "more stuff will appear here" and don't move off my current document until it does (sounds a lot like a 204).

                    The server just puts out the archives as needed. No playing with etags or mutability (every uri is always infinitely cachable). 

                    The protocol makes even more sense when you consider dropping long polling on top of it (get me this thing that may or may not exist). 

                    Cheers,

                    Greg

                    On Fri, Jun 28, 2013 at 5:32 PM, Markus Lanthaler <markus.lanthaler@...> wrote:
                     

                    On Friday, June 28, 2013 3:19 PM, Greg Young wrote


                    > btw in doing some research it would appear that RFC 5005 allows
                    > for either in the areas where reading of feeds is actually discussed.

                    > This process
                    > should be repeated recursively until the client encounters a prev-
                    > archive link relation that has been processed (the end of the archive
                    > is indicated by a missing prev-archive link relation) or an error is
                    > encountered.
                    >
                    > prev/next reading is not discussed but would seem logical to work in the
                    > same way. I am unsure why it would be "broken" as you say to return an
                    > error. Can link me to the relevant specification?

                    I didn't say that a spec prohibits it, how could it? You have to expect
                    broken links everywhere. I just said I would assume that your implementation
                    is broken because it creates links that lead to a 404.

                    Errors shouldn't control the *normal* control flow, they should be
                    exceptions.

                    --
                    Markus Lanthaler
                    @markuslanthaler




                    --
                    Le doute n'est pas une condition agréable, mais la certitude est absurde.
                  • Greg Young
                    btw just to add to this. The specification contradicts itself. If I have to adjust a prev link on the archive ... How can: The requirement that archive
                    Message 9 of 23 , Jun 28, 2013
                    View Source
                    • 0 Attachment
                      btw just to add to this. The specification contradicts itself. If I
                      have to adjust a prev link on the archive ...

                      How can:

                      " The requirement that archive documents be stable allows clients to

                      safely assume that if they have retrieved one in the past, it will
                      not meaningfully change in the future. "

                      I would tend to read this as immutability.

                      Is changing a prev link not a meaningful change? I would consider it
                      very meaningful.

                      Cheers,

                      Greg


                      On Fri, Jun 28, 2013 at 5:45 PM, Greg Young <gregoryyoung1@...> wrote:
                      >
                      > And doing it your way requires every uri we create to be put out twice (once as mutable and uncachable once as immutable and infinitely cachable).
                      >
                      > Take archives. My current archive is therefor mutable. I have to rewrite the archive every time I add a new one. This sounds like a fairly complicated protocol as opposed to pre-writing what the next uri will be and giving some sort of status when it is not yet in existence. This becomes even more so if my archives are say monthly and widely distributed. This is especially true for us as our data is already immutable (prev/next were actually the wrong choice, we should be using archive)
                      >
                      > Your client just has to say "more stuff will appear here" and don't move off my current document until it does (sounds a lot like a 204).
                      >
                      > The server just puts out the archives as needed. No playing with etags or mutability (every uri is always infinitely cachable).
                      >
                      > The protocol makes even more sense when you consider dropping long polling on top of it (get me this thing that may or may not exist).
                      >
                      > Cheers,
                      >
                      > Greg
                      >
                      > On Fri, Jun 28, 2013 at 5:32 PM, Markus Lanthaler <markus.lanthaler@...> wrote:
                      >>
                      >>
                      >>
                      >> On Friday, June 28, 2013 3:19 PM, Greg Young wrote
                      >>
                      >>
                      >> > btw in doing some research it would appear that RFC 5005 allows
                      >> > for either in the areas where reading of feeds is actually discussed.
                      >> >
                      >> > This process
                      >> > should be repeated recursively until the client encounters a prev-
                      >> > archive link relation that has been processed (the end of the archive
                      >> > is indicated by a missing prev-archive link relation) or an error is
                      >> > encountered.
                      >> >
                      >> > prev/next reading is not discussed but would seem logical to work in the
                      >> > same way. I am unsure why it would be "broken" as you say to return an
                      >> > error. Can link me to the relevant specification?
                      >>
                      >> I didn't say that a spec prohibits it, how could it? You have to expect
                      >> broken links everywhere. I just said I would assume that your implementation
                      >> is broken because it creates links that lead to a 404.
                      >>
                      >> Errors shouldn't control the *normal* control flow, they should be
                      >> exceptions.
                      >>
                      >> --
                      >> Markus Lanthaler
                      >> @markuslanthaler
                      >>
                      >>
                      >
                      >
                      >
                      >
                      > --
                      > Le doute n'est pas une condition agréable, mais la certitude est absurde.




                      --
                      Le doute n'est pas une condition agréable, mais la certitude est absurde.
                    • Erik Wilde
                      could a link hint help? saying this will eventually work but don t expect it to work before ... ? cheers, dret. ... could a link hint help? saying this will
                      Message 10 of 23 , Jun 28, 2013
                      View Source
                      • 0 Attachment
                        could a link hint help? saying "this will eventually work but don't expect it to work before ..."? cheers, dret.

                        On Jun 28, 2013, at 7:45, Greg Young <gregoryyoung1@...> wrote:

                        And doing it your way requires every uri we create to be put out twice (once as mutable and uncachable once as immutable and infinitely cachable).

                        Take archives. My current archive is therefor mutable. I have to rewrite the archive every time I add a new one. This sounds like a fairly complicated protocol as opposed to pre-writing what the next uri will be and giving some sort of status when it is not yet in existence. This becomes even more so if my archives are say monthly and widely distributed. This is especially true for us as our data is already immutable (prev/next were actually the wrong choice, we should be using archive)

                        Your client just has to say "more stuff will appear here" and don't move off my current document until it does (sounds a lot like a 204).

                        The server just puts out the archives as needed. No playing with etags or mutability (every uri is always infinitely cachable). 

                        The protocol makes even more sense when you consider dropping long polling on top of it (get me this thing that may or may not exist). 

                        Cheers,

                        Greg

                        On Fri, Jun 28, 2013 at 5:32 PM, Markus Lanthaler <markus.lanthaler@...> wrote:
                         

                        On Friday, June 28, 2013 3:19 PM, Greg Young wrote


                        > btw in doing some research it would appear that RFC 5005 allows
                        > for either in the areas where reading of feeds is actually discussed.

                        > This process
                        > should be repeated recursively until the client encounters a prev-
                        > archive link relation that has been processed (the end of the archive
                        > is indicated by a missing prev-archive link relation) or an error is
                        > encountered.
                        >
                        > prev/next reading is not discussed but would seem logical to work in the
                        > same way. I am unsure why it would be "broken" as you say to return an
                        > error. Can link me to the relevant specification?

                        I didn't say that a spec prohibits it, how could it? You have to expect
                        broken links everywhere. I just said I would assume that your implementation
                        is broken because it creates links that lead to a 404.

                        Errors shouldn't control the *normal* control flow, they should be
                        exceptions.

                        --
                        Markus Lanthaler
                        @markuslanthaler




                        --
                        Le doute n'est pas une condition agréable, mais la certitude est absurde.
                      • Greg Young
                        we dont know when it will work. ... -- Le doute n est pas une condition agréable, mais la certitude est absurde. we dont know when it will work. On Fri, Jun
                        Message 11 of 23 , Jun 28, 2013
                        View Source
                        • 0 Attachment
                          we dont know when it will work.

                          On Fri, Jun 28, 2013 at 6:14 PM, Erik Wilde <dret@...> wrote:
                          could a link hint help? saying "this will eventually work but don't expect it to work before ..."? cheers, dret.

                          On Jun 28, 2013, at 7:45, Greg Young <gregoryyoung1@...> wrote:



                          And doing it your way requires every uri we create to be put out twice (once as mutable and uncachable once as immutable and infinitely cachable).

                          Take archives. My current archive is therefor mutable. I have to rewrite the archive every time I add a new one. This sounds like a fairly complicated protocol as opposed to pre-writing what the next uri will be and giving some sort of status when it is not yet in existence. This becomes even more so if my archives are say monthly and widely distributed. This is especially true for us as our data is already immutable (prev/next were actually the wrong choice, we should be using archive)

                          Your client just has to say "more stuff will appear here" and don't move off my current document until it does (sounds a lot like a 204).

                          The server just puts out the archives as needed. No playing with etags or mutability (every uri is always infinitely cachable). 

                          The protocol makes even more sense when you consider dropping long polling on top of it (get me this thing that may or may not exist). 

                          Cheers,

                          Greg

                          On Fri, Jun 28, 2013 at 5:32 PM, Markus Lanthaler <markus.lanthaler@...> wrote:
                           

                          On Friday, June 28, 2013 3:19 PM, Greg Young wrote


                          > btw in doing some research it would appear that RFC 5005 allows
                          > for either in the areas where reading of feeds is actually discussed.

                          > This process
                          > should be repeated recursively until the client encounters a prev-
                          > archive link relation that has been processed (the end of the archive
                          > is indicated by a missing prev-archive link relation) or an error is
                          > encountered.
                          >
                          > prev/next reading is not discussed but would seem logical to work in the
                          > same way. I am unsure why it would be "broken" as you say to return an
                          > error. Can link me to the relevant specification?

                          I didn't say that a spec prohibits it, how could it? You have to expect
                          broken links everywhere. I just said I would assume that your implementation
                          is broken because it creates links that lead to a 404.

                          Errors shouldn't control the *normal* control flow, they should be
                          exceptions.

                          --
                          Markus Lanthaler
                          @markuslanthaler




                          --
                          Le doute n'est pas une condition agréable, mais la certitude est absurde.





                          --
                          Le doute n'est pas une condition agréable, mais la certitude est absurde.
                        • Erik Wilde
                          i guess then there s little to communicate in a link hint (other than a very vague maybe don t try before X ). but if the links are expected to work at some
                          Message 12 of 23 , Jun 28, 2013
                          View Source
                          • 0 Attachment
                            i guess then there's little to communicate in a link hint (other than a
                            very vague "maybe don't try before X"). but if the links are expected to
                            work at some predictable time (a daily newsletter, for example), then i
                            guess you could you a link hint to inform clients about this.

                            On 2013-06-28 08:23 , Greg Young wrote:
                            > we dont know when it will work.
                            >
                            > On Fri, Jun 28, 2013 at 6:14 PM, Erik Wilde <dret@...
                            > <mailto:dret@...>> wrote:
                            >
                            > could a link hint help? saying "this will eventually work but don't
                            > expect it to work before ..."? cheers, dret.
                            >
                            > On Jun 28, 2013, at 7:45, Greg Young <gregoryyoung1@...
                            > <mailto:gregoryyoung1@...>> wrote:
                            >
                            >>
                            >>
                            >> And doing it your way requires every uri we create to be put out
                            >> twice (once as mutable and uncachable once as immutable and
                            >> infinitely cachable).
                            >>
                            >> Take archives. My current archive is therefor mutable. I have to
                            >> rewrite the archive every time I add a new one. This sounds like a
                            >> fairly complicated protocol as opposed to pre-writing what the
                            >> next uri will be and giving some sort of status when it is not yet
                            >> in existence. This becomes even more so if my archives are say
                            >> monthly and widely distributed. This is especially true for us as
                            >> our data is already immutable (prev/next were actually the wrong
                            >> choice, we should be using archive)
                            >>
                            >> Your client just has to say "more stuff will appear here" and
                            >> don't move off my current document until it does (sounds a lot
                            >> like a 204).
                            >>
                            >> The server just puts out the archives as needed. No playing with
                            >> etags or mutability (every uri is always infinitely cachable).
                            >>
                            >> The protocol makes even more sense when you consider dropping long
                            >> polling on top of it (get me this thing that may or may not exist).
                            >>
                            >> Cheers,
                            >>
                            >> Greg
                            >>
                            >> On Fri, Jun 28, 2013 at 5:32 PM, Markus Lanthaler
                            >> <markus.lanthaler@... <mailto:markus.lanthaler@...>> wrote:
                            >>
                            >> __
                            >>
                            >> On Friday, June 28, 2013 3:19 PM, Greg Young wrote
                            >>
                            >>
                            >> > btw in doing some research it would appear that RFC 5005 allows
                            >> > for either in the areas where reading of feeds is actually
                            >> discussed.
                            >> >
                            >> > This process
                            >> > should be repeated recursively until the client encounters a
                            >> prev-
                            >> > archive link relation that has been processed (the end of
                            >> the archive
                            >> > is indicated by a missing prev-archive link relation) or an
                            >> error is
                            >> > encountered.
                            >> >
                            >> > prev/next reading is not discussed but would seem logical to
                            >> work in the
                            >> > same way. I am unsure why it would be "broken" as you say to
                            >> return an
                            >> > error. Can link me to the relevant specification?
                            >>
                            >> I didn't say that a spec prohibits it, how could it? You have
                            >> to expect
                            >> broken links everywhere. I just said I would assume that your
                            >> implementation
                            >> is broken because it creates links that lead to a 404.
                            >>
                            >> Errors shouldn't control the *normal* control flow, they should be
                            >> exceptions.
                            >>
                            >> --
                            >> Markus Lanthaler
                            >> @markuslanthaler
                            >>
                            >>
                            >>
                            >>
                            >> --
                            >> Le doute n'est pas une condition agréable, mais la certitude est
                            >> absurde.
                            >>
                            >>
                            >>
                            >
                            >
                            >
                            >
                            > --
                            > Le doute n'est pas une condition agréable, mais la certitude est absurde.

                            --
                            erik wilde | mailto:dret@... - tel:+1-510-2061079 |
                            | UC Berkeley - School of Information (ISchool) |
                            | http://dret.net/netdret http://twitter.com/dret |
                          • Greg Young
                            not predictable (its an event stream) it could be 1ms or 30 days. we have switched to archive-prev (our stream is complete and non-lossy) but it really feels
                            Message 13 of 23 , Jun 28, 2013
                            View Source
                            • 0 Attachment
                              not predictable (its an event stream) it could be 1ms or 30 days.

                              we have switched to archive-prev (our stream is complete and
                              non-lossy) but it really feels like an immutable prev/next is needed
                              to be standardized.

                              we figured a 204 "don't change your document" + retry fit the bill.
                              maybe not ...

                              As of now there is no good way to do what we do (non-lossy stream)
                              under 5005. As such we will have to just do something I guess and then
                              try to get a spec change. The changes are not huge and I think are in
                              the spirit of the spec (weird how they want mutability and
                              immutability at the same time).

                              Greg

                              On Fri, Jun 28, 2013 at 6:44 PM, Erik Wilde <dret@...> wrote:
                              > i guess then there's little to communicate in a link hint (other than a very
                              > vague "maybe don't try before X"). but if the links are expected to work at
                              > some predictable time (a daily newsletter, for example), then i guess you
                              > could you a link hint to inform clients about this.
                              >
                              >
                              > On 2013-06-28 08:23 , Greg Young wrote:
                              >>
                              >> we dont know when it will work.
                              >>
                              >> On Fri, Jun 28, 2013 at 6:14 PM, Erik Wilde <dret@...
                              >> <mailto:dret@...>> wrote:
                              >>
                              >> could a link hint help? saying "this will eventually work but don't
                              >> expect it to work before ..."? cheers, dret.
                              >>
                              >> On Jun 28, 2013, at 7:45, Greg Young <gregoryyoung1@...
                              >> <mailto:gregoryyoung1@...>> wrote:
                              >>
                              >>>
                              >>>
                              >>> And doing it your way requires every uri we create to be put out
                              >>> twice (once as mutable and uncachable once as immutable and
                              >>> infinitely cachable).
                              >>>
                              >>> Take archives. My current archive is therefor mutable. I have to
                              >>> rewrite the archive every time I add a new one. This sounds like a
                              >>> fairly complicated protocol as opposed to pre-writing what the
                              >>> next uri will be and giving some sort of status when it is not yet
                              >>> in existence. This becomes even more so if my archives are say
                              >>> monthly and widely distributed. This is especially true for us as
                              >>> our data is already immutable (prev/next were actually the wrong
                              >>> choice, we should be using archive)
                              >>>
                              >>> Your client just has to say "more stuff will appear here" and
                              >>> don't move off my current document until it does (sounds a lot
                              >>> like a 204).
                              >>>
                              >>> The server just puts out the archives as needed. No playing with
                              >>> etags or mutability (every uri is always infinitely cachable).
                              >>>
                              >>> The protocol makes even more sense when you consider dropping long
                              >>> polling on top of it (get me this thing that may or may not exist).
                              >>>
                              >>> Cheers,
                              >>>
                              >>> Greg
                              >>>
                              >>> On Fri, Jun 28, 2013 at 5:32 PM, Markus Lanthaler
                              >>> <markus.lanthaler@... <mailto:markus.lanthaler@...>> wrote:
                              >>>
                              >>> __
                              >>>
                              >>>
                              >>> On Friday, June 28, 2013 3:19 PM, Greg Young wrote
                              >>>
                              >>>
                              >>> > btw in doing some research it would appear that RFC 5005 allows
                              >>> > for either in the areas where reading of feeds is actually
                              >>> discussed.
                              >>> >
                              >>> > This process
                              >>> > should be repeated recursively until the client encounters a
                              >>> prev-
                              >>> > archive link relation that has been processed (the end of
                              >>> the archive
                              >>> > is indicated by a missing prev-archive link relation) or an
                              >>> error is
                              >>> > encountered.
                              >>> >
                              >>> > prev/next reading is not discussed but would seem logical to
                              >>> work in the
                              >>> > same way. I am unsure why it would be "broken" as you say to
                              >>> return an
                              >>> > error. Can link me to the relevant specification?
                              >>>
                              >>> I didn't say that a spec prohibits it, how could it? You have
                              >>> to expect
                              >>> broken links everywhere. I just said I would assume that your
                              >>> implementation
                              >>> is broken because it creates links that lead to a 404.
                              >>>
                              >>> Errors shouldn't control the *normal* control flow, they should
                              >>> be
                              >>> exceptions.
                              >>>
                              >>> --
                              >>> Markus Lanthaler
                              >>> @markuslanthaler
                              >>>
                              >>>
                              >>>
                              >>>
                              >>> --
                              >>> Le doute n'est pas une condition agréable, mais la certitude est
                              >>> absurde.
                              >>>
                              >>>
                              >>>
                              >>
                              >>
                              >>
                              >>
                              >>
                              >> --
                              >> Le doute n'est pas une condition agréable, mais la certitude est absurde.
                              >
                              >
                              > --
                              > erik wilde | mailto:dret@... - tel:+1-510-2061079 |
                              > | UC Berkeley - School of Information (ISchool) |
                              > | http://dret.net/netdret http://twitter.com/dret |



                              --
                              Le doute n'est pas une condition agréable, mais la certitude est absurde.
                            • Erik Mogensen
                              ... also use the prev link of the first page of a feed to link to the page containing the items that will be . We find that it s an extremely resilient way
                              Message 14 of 23 , Jun 28, 2013
                              View Source
                              • 0 Attachment

                                On Fri, Jun 28, 2013 at 12:16 PM, Greg Young <gregoryyoung1@...> wrote:
                                 

                                We have a mature atom implementation for distribution of event streams
                                and as always the devil is in the details...

                                The issue revolves around prev links. At some point on an atom feed
                                you will hit the current item, prev will point to the next item that
                                does not yet exist. Most implementations I can find 404 this uri and
                                the client has the logic to keep polling the 404. This seems off to
                                me, I would expect this to be a 204. Then if I change my uri structure
                                it becomes a 404, otherwise it would seem that if I change my uri
                                structure all prev uris would need to redirect to the new ones (which
                                would be a rather large undertaking).

                                I couldn't find any details on this in relevant specs, has anyone seen this?

                                Greg, I'd just like to explain how we do more or less the same thing.  We also use the prev link of the "first page" of a feed to link to the page containing the items that "will be".  We find that it's an extremely resilient way of dissipating information about things that fit in atom feeds, and a relatively efficient system too.

                                Infinitely cacheable resources are nice, but instead of 404, or 204 and so on, we expose an empty feed document, with a next link but no prev link.  When things happen, the page (when refreshed) contains one or more entries, and becomes cacheable.

                                Our "head" document is very small; contains no real data so it's easy to generate; supports cache validation, and if it's incorrectly cached as fresh by an intermediary or a caching client, it only impedes the ability of clients to advance to the "newer" entries.  Eventually, they will advance to the next state. 

                                If I were to do this again, I might consider using 302 some other temporary redirect, instead of the 404 or the empty feed; redirect back to the current (infinitely cacheable) first page.  When a new item appears, the 302 would turn into the "new" first page" a 200 with new next/prev links, 

                                I wonder if "Cache-Control" or "Retry-After" would be useful for 302 responses, to ask clients to back off under heavy load.
                                --
                                -mogsie-
                              • Greg Young
                                We have considered this empty feed page as well! We were thinking its slightly more efficient (and understandable) to have a status code but maybe not
                                Message 15 of 23 , Jun 28, 2013
                                View Source
                                • 0 Attachment
                                  We have considered this empty feed page as well! We were thinking its
                                  slightly more efficient (and understandable) to have a status code but
                                  maybe not (thinking client logic).

                                  If I were to do this again, I might consider using 302 some other
                                  temporary redirect, instead of the 404 or the empty feed; redirect
                                  back to the current (infinitely cacheable) first page. When a new
                                  item appears, the 302 would turn into the "new" first page" a 200 with
                                  new next/prev links,

                                  I am curious why not a 204. It would seem to be a reasonable fit here
                                  "keep your doc, retry after ..."

                                  Cheers,

                                  Greg

                                  On Fri, Jun 28, 2013 at 7:09 PM, Erik Mogensen <erik@...> wrote:
                                  >
                                  > On Fri, Jun 28, 2013 at 12:16 PM, Greg Young <gregoryyoung1@...>
                                  > wrote:
                                  >>
                                  >>
                                  >>
                                  >> We have a mature atom implementation for distribution of event streams
                                  >> and as always the devil is in the details...
                                  >>
                                  >> The issue revolves around prev links. At some point on an atom feed
                                  >> you will hit the current item, prev will point to the next item that
                                  >> does not yet exist. Most implementations I can find 404 this uri and
                                  >> the client has the logic to keep polling the 404. This seems off to
                                  >> me, I would expect this to be a 204. Then if I change my uri structure
                                  >> it becomes a 404, otherwise it would seem that if I change my uri
                                  >> structure all prev uris would need to redirect to the new ones (which
                                  >> would be a rather large undertaking).
                                  >>
                                  >> I couldn't find any details on this in relevant specs, has anyone seen
                                  >> this?
                                  >>
                                  > Greg, I'd just like to explain how we do more or less the same thing. We
                                  > also use the prev link of the "first page" of a feed to link to the page
                                  > containing the items that "will be". We find that it's an extremely
                                  > resilient way of dissipating information about things that fit in atom
                                  > feeds, and a relatively efficient system too.
                                  >
                                  > Infinitely cacheable resources are nice, but instead of 404, or 204 and so
                                  > on, we expose an empty feed document, with a next link but no prev link.
                                  > When things happen, the page (when refreshed) contains one or more entries,
                                  > and becomes cacheable.
                                  >
                                  > Our "head" document is very small; contains no real data so it's easy to
                                  > generate; supports cache validation, and if it's incorrectly cached as fresh
                                  > by an intermediary or a caching client, it only impedes the ability of
                                  > clients to advance to the "newer" entries. Eventually, they will advance to
                                  > the next state.
                                  >
                                  > If I were to do this again, I might consider using 302 some other temporary
                                  > redirect, instead of the 404 or the empty feed; redirect back to the current
                                  > (infinitely cacheable) first page. When a new item appears, the 302 would
                                  > turn into the "new" first page" a 200 with new next/prev links,
                                  >
                                  > I wonder if "Cache-Control" or "Retry-After" would be useful for 302
                                  > responses, to ask clients to back off under heavy load.
                                  > --
                                  > -mogsie-



                                  --
                                  Le doute n'est pas une condition agréable, mais la certitude est absurde.
                                • Erik Mogensen
                                  ... We were probably thinking that the empty page would fit nicely within an IP packet anyway, and could easily support ETag validation, and provide
                                  Message 16 of 23 , Jun 28, 2013
                                  View Source
                                  • 0 Attachment
                                    On Fri, Jun 28, 2013 at 6:14 PM, Greg Young <gregoryyoung1@...> wrote:
                                     

                                    We have considered this empty feed page as well! We were thinking its
                                    slightly more efficient (and understandable) to have a status code but
                                    maybe not (thinking client logic).

                                    We were probably thinking that the empty page would fit nicely within an IP packet anyway, and could easily support ETag validation, and provide information about the current state "there's nothing to process now".

                                    I am curious why not a 204. It would seem to be a reasonable fit here
                                    "keep your doc, retry after ..."


                                    Hmm. after reading up on RFC-2616 I'd agree with you.  My understanding of 204 means that you should "stay where you were" and then I guess the client would eventually retry.

                                    RFC2616, section 10.2.5 "204 NO CONTENT" says: "If the client is a user agent, it SHOULD NOT change its document view from that which caused the request to be sent" — This should cause the same or a very similar outcome, I guess.

                                    I guess we should update our client to react accordingly when we encounter a 204. :-)
                                    --
                                    -mogsie-
                                  • Ben Niven-Jenkins
                                    ... Why not use feed paging and use prev-archive to indicate the previous page and keep your main feed as the current page. Client just polls the current
                                    Message 17 of 23 , Jun 28, 2013
                                    View Source
                                    • 0 Attachment
                                      On 28 Jun 2013, at 11:16, Greg Young <gregoryyoung1@...> wrote:

                                      > We have a mature atom implementation for distribution of event streams
                                      > and as always the devil is in the details...
                                      >
                                      > The issue revolves around prev links. At some point on an atom feed
                                      > you will hit the current item, prev will point to the next item that
                                      > does not yet exist.

                                      Why not use feed paging and use prev-archive to indicate the previous page and keep your "main" feed as the current page.

                                      Client just polls the current page and if it needs to go further back in time it follows prev-archive links until it finds what it wants.

                                      You then avoid the problem of links to things that don't exist or updating already archived pages.

                                      Ben
                                    • Greg Young
                                      Our whole feed is nonlossy and immutable. ... -- Le doute n est pas une condition agréable, mais la certitude est absurde. Our whole feed is nonlossy and
                                      Message 18 of 23 , Jun 28, 2013
                                      View Source
                                      • 0 Attachment
                                        Our whole feed is nonlossy and immutable.

                                        On Friday, June 28, 2013, Ben Niven-Jenkins wrote:


                                        On 28 Jun 2013, at 11:16, Greg Young <gregoryyoung1@...> wrote:

                                        > We have a mature atom implementation for distribution of event streams
                                        > and as always the devil is in the details...
                                        >
                                        > The issue revolves around prev links. At some point on an atom feed
                                        > you will hit the current item, prev will point to the next item that
                                        > does not yet exist.

                                        Why not use feed paging and use prev-archive to indicate the previous page and keep your "main" feed as the current page.

                                        Client just polls the current page and if it needs to go further back in time it follows prev-archive links until it finds what it wants.

                                        You then avoid the problem of links to things that don't exist or updating already archived pages.

                                        Ben


                                        --
                                        Le doute n'est pas une condition agréable, mais la certitude est absurde.
                                      • Ben Niven-Jenkins
                                        ... Can you expand on what you mean by this. Feed archiving doesn t lose information and doesn t require changes to a page once it is first generated. Or do
                                        Message 19 of 23 , Jun 28, 2013
                                        View Source
                                        • 0 Attachment


                                          On 28 Jun 2013, at 18:32, Greg Young <gregoryyoung1@...> wrote:

                                          Our whole feed is nonlossy and immutable.

                                          Can you expand on what you mean by this. Feed archiving doesn't lose information and doesn't require changes to a page once it is first generated. 

                                          Or do you mean the client cannot handle the content of the URI that represents the entry point into the feed changing and expects the content of that URI to stay always constant so the clients needs to page "forward in time" to find the latest entries?

                                          Ben

                                          On Friday, June 28, 2013, Ben Niven-Jenkins wrote:


                                          On 28 Jun 2013, at 11:16, Greg Young <gregoryyoung1@...> wrote:

                                          > We have a mature atom implementation for distribution of event streams
                                          > and as always the devil is in the details...
                                          >
                                          > The issue revolves around prev links. At some point on an atom feed
                                          > you will hit the current item, prev will point to the next item that
                                          > does not yet exist.

                                          Why not use feed paging and use prev-archive to indicate the previous page and keep your "main" feed as the current page.

                                          Client just polls the current page and if it needs to go further back in time it follows prev-archive links until it finds what it wants.

                                          You then avoid the problem of links to things that don't exist or updating already archived pages.

                                          Ben


                                          --
                                          Le doute n'est pas une condition agréable, mais la certitude est absurde.
                                        • Greg Young
                                          We could do all over prev next even with no archiving. Every uri we create is immutable. Spec defines additional logic needed for prev/next which is unneeded
                                          Message 20 of 23 , Jun 28, 2013
                                          View Source
                                          • 0 Attachment
                                            We could do all over prev next even with no archiving. Every uri we create is immutable. Spec defines additional logic needed for prev/next which is unneeded for us so archive gives us a simpler client API (also explicit that we are non lossy)

                                            On Friday, June 28, 2013, Ben Niven-Jenkins wrote:


                                            On 28 Jun 2013, at 18:32, Greg Young <gregoryyoung1@...> wrote:

                                            Our whole feed is nonlossy and immutable.

                                            Can you expand on what you mean by this. Feed archiving doesn't lose information and doesn't require changes to a page once it is first generated. 

                                            Or do you mean the client cannot handle the content of the URI that represents the entry point into the feed changing and expects the content of that URI to stay always constant so the clients needs to page "forward in time" to find the latest entries?

                                            Ben

                                            On Friday, June 28, 2013, Ben Niven-Jenkins wrote:


                                            On 28 Jun 2013, at 11:16, Greg Young <gregoryyoung1@...> wrote:

                                            > We have a mature atom implementation for distribution of event streams
                                            > and as always the devil is in the details...
                                            >
                                            > The issue revolves around prev links. At some point on an atom feed
                                            > you will hit the current item, prev will point to the next item that
                                            > does not yet exist.

                                            Why not use feed paging and use prev-archive to indicate the previous page and keep your "main" feed as the current page.

                                            Client just polls the current page and if it needs to go further back in time it follows prev-archive links until it finds what it wants.

                                            You then avoid the problem of links to things that don't exist or updating already archived pages.

                                            Ben


                                            --
                                            Le doute n'est pas une condition agréable, mais la certitude est absurde.


                                            --
                                            Le doute n'est pas une condition agréable, mais la certitude est absurde.
                                          • Greg Young
                                            To be clear we want a cross of prev and archive prev as they were obviously designed for mutable feeds. ... -- Le doute n est pas une condition agréable, mais
                                            Message 21 of 23 , Jun 28, 2013
                                            View Source
                                            • 0 Attachment
                                              To be clear we want a cross of prev and archive prev as they were obviously designed for mutable feeds.

                                              On Friday, June 28, 2013, Greg Young wrote:
                                              We could do all over prev next even with no archiving. Every uri we create is immutable. Spec defines additional logic needed for prev/next which is unneeded for us so archive gives us a simpler client API (also explicit that we are non lossy)

                                              On Friday, June 28, 2013, Ben Niven-Jenkins wrote:


                                              On 28 Jun 2013, at 18:32, Greg Young <gregoryyoung1@...> wrote:

                                              Our whole feed is nonlossy and immutable.

                                              Can you expand on what you mean by this. Feed archiving doesn't lose information and doesn't require changes to a page once it is first generated. 

                                              Or do you mean the client cannot handle the content of the URI that represents the entry point into the feed changing and expects the content of that URI to stay always constant so the clients needs to page "forward in time" to find the latest entries?

                                              Ben

                                              On Friday, June 28, 2013, Ben Niven-Jenkins wrote:


                                              On 28 Jun 2013, at 11:16, Greg Young <gregoryyoung1@...> wrote:

                                              > We have a mature atom implementation for distribution of event streams
                                              > and as always the devil is in the details...
                                              >
                                              > The issue revolves around prev links. At some point on an atom feed
                                              > you will hit the current item, prev will point to the next item that
                                              > does not yet exist.

                                              Why not use feed paging and use prev-archive to indicate the previous page and keep your "main" feed as the current page.

                                              Client just polls the current page and if it needs to go further back in time it follows prev-archive links until it finds what it wants.

                                              You then avoid the problem of links to things that don't exist or updating already archived pages.

                                              Ben


                                              --
                                              Le doute n'est pas une condition agréable, mais la certitude est absurde.


                                              --
                                              Le doute n'est pas une condition agréable, mais la certitude est absurde.


                                              --
                                              Le doute n'est pas une condition agréable, mais la certitude est absurde.
                                            Your message has been successfully submitted and would be delivered to recipients shortly.