Loading ...
Sorry, an error occurred while loading the content.

Atom and prev links

Expand Messages
  • Greg Young
    We have a mature atom implementation for distribution of event streams and as always the devil is in the details... The issue revolves around prev links. At
    Message 1 of 23 , Jun 28, 2013
    • 0 Attachment
      We have a mature atom implementation for distribution of event streams
      and as always the devil is in the details...

      The issue revolves around prev links. At some point on an atom feed
      you will hit the current item, prev will point to the next item that
      does not yet exist. Most implementations I can find 404 this uri and
      the client has the logic to keep polling the 404. This seems off to
      me, I would expect this to be a 204. Then if I change my uri structure
      it becomes a 404, otherwise it would seem that if I change my uri
      structure all prev uris would need to redirect to the new ones (which
      would be a rather large undertaking).

      I couldn't find any details on this in relevant specs, has anyone seen this?

      Cheers,

      Greg

      --
      Le doute n'est pas une condition agréable, mais la certitude est absurde.
    • Markus Lanthaler
      ... Why don t you just hide the next/prev link till the next/prev resource really exists? If really necessary, the client can keep polling the last resource it
      Message 2 of 23 , Jun 28, 2013
      • 0 Attachment
        On Friday, June 28, 2013 12:16 PM, Greg Young wrote:
        > The issue revolves around prev links. At some point on an atom feed
        > you will hit the current item, prev will point to the next item that
        > does not yet exist. Most implementations I can find 404 this uri and
        > the client has the logic to keep polling the 404. This seems off to
        > me, I would expect this to be a 204. Then if I change my uri structure
        > it becomes a 404, otherwise it would seem that if I change my uri
        > structure all prev uris would need to redirect to the new ones (which
        > would be a rather large undertaking).

        Why don't you just hide the next/prev link till the next/prev resource
        really exists? If really necessary, the client can keep polling the last
        resource it encountered till the next/prev link becomes available. This also
        means that you can decide how to construct the URL of the new element as
        soon as it is created instead of relying on, e.g., an autoincrement.

        Did I miss something?


        --
        Markus Lanthaler
        @markuslanthaler
      • Greg Young
        Actually we have a pretty simple reason for this caching. If we were to hide the prev that uri would have to be non-cachable. If we put it there (we are
        Message 3 of 23 , Jun 28, 2013
        • 0 Attachment
          Actually we have a pretty simple reason for this caching.

          If we were to hide the prev that uri would have to be non-cachable. If we put it there (we are forward only) then every valid uri we have is infinitely cachable. By predefining the next only our non-existant uri is non-cachable.

          Though keeping the last existing one as non-cachable could also solve this and maybe its a better overall way.

          Cheers,

          Greg

          On Fri, Jun 28, 2013 at 1:47 PM, Markus Lanthaler <markus.lanthaler@...> wrote:
           

          On Friday, June 28, 2013 12:16 PM, Greg Young wrote:
          > The issue revolves around prev links. At some point on an atom feed
          > you will hit the current item, prev will point to the next item that
          > does not yet exist. Most implementations I can find 404 this uri and
          > the client has the logic to keep polling the 404. This seems off to
          > me, I would expect this to be a 204. Then if I change my uri structure
          > it becomes a 404, otherwise it would seem that if I change my uri
          > structure all prev uris would need to redirect to the new ones (which
          > would be a rather large undertaking).

          Why don't you just hide the next/prev link till the next/prev resource
          really exists? If really necessary, the client can keep polling the last
          resource it encountered till the next/prev link becomes available. This also
          means that you can decide how to construct the URL of the new element as
          soon as it is created instead of relying on, e.g., an autoincrement.

          Did I miss something?

          --
          Markus Lanthaler
          @markuslanthaler




          --
          Le doute n'est pas une condition agréable, mais la certitude est absurde.
        • Markus Lanthaler
          ... Well, a 404 can be cached as well, so it s not that much of a difference. Just use an Etag or something similar for the last page and drop it as soon as
          Message 4 of 23 , Jun 28, 2013
          • 0 Attachment
            On Friday, June 28, 2013 12:58 PM, Greg Young wrote:
            > Actually we have a pretty simple reason for this caching.
            >
            > If we were to hide the prev that uri would have to be non-cachable.
            > If we put it there (we are forward only) then every valid uri we
            > have is infinitely cachable. By predefining the next only our
            > non-existant uri is non-cachable.

            Well, a 404 can be cached as well, so it's not that much of a difference.
            Just use an Etag or something similar for the last page and drop it as soon
            as you put the link in.

            If I would implement a client, I wouldn't keep polling the 404. I would just
            assume your feed is broken.


            --
            Markus Lanthaler
            @markuslanthaler
          • Greg Young
            And if I returned a 204? On Fri, Jun 28, 2013 at 3:04 PM, Markus Lanthaler ... -- Le doute n est pas une condition agréable, mais la certitude est absurde.
            Message 5 of 23 , Jun 28, 2013
            • 0 Attachment
              And if I returned a 204?

              On Fri, Jun 28, 2013 at 3:04 PM, Markus Lanthaler <markus.lanthaler@...> wrote:
               

              On Friday, June 28, 2013 12:58 PM, Greg Young wrote:
              > Actually we have a pretty simple reason for this caching.
              >
              > If we were to hide the prev that uri would have to be non-cachable.
              > If we put it there (we are forward only) then every valid uri we
              > have is infinitely cachable. By predefining the next only our
              > non-existant uri is non-cachable.

              Well, a 404 can be cached as well, so it's not that much of a difference.
              Just use an Etag or something similar for the last page and drop it as soon
              as you put the link in.

              If I would implement a client, I wouldn't keep polling the 404. I would just
              assume your feed is broken.

              --
              Markus Lanthaler
              @markuslanthaler




              --
              Le doute n'est pas une condition agréable, mais la certitude est absurde.
            • Markus Lanthaler
              ... No, the status code that would make most sense IMO is 202.. but I would still suggest to just hide the link till the resource gets created. -- Markus
              Message 6 of 23 , Jun 28, 2013
              • 0 Attachment
                On Friday, June 28, 2013 2:10 PM, Greg Young wrote:
                > On Fri, Jun 28, 2013 at 3:04 PM, Markus Lanthaler wrote:
                >> If I would implement a client, I wouldn't keep polling the 404.
                >> I would just assume your feed is broken.
                >
                >And if I returned a 204?

                No, the status code that would make "most" sense IMO is 202.. but I would
                still suggest to just hide the link till the resource gets created.


                --
                Markus Lanthaler
                @markuslanthaler
              • Greg Young
                btw in doing some research it would appear that RFC 5005 allows for either in the areas where reading of feeds is actually discussed. This process should be
                Message 7 of 23 , Jun 28, 2013
                • 0 Attachment
                  btw in doing some research it would appear that RFC 5005 allows for either in the areas where reading of feeds is actually discussed.
                   
                  This process
                     should be repeated recursively until the client encounters a prev-
                     archive link relation that has been processed (the end of the archive
                     is indicated by a missing prev-archive link relation) or an error is
                     encountered.

                  prev/next reading is not discussed but would seem logical to work in the same way. I am unsure why it would be "broken" as you say to return an error. Can link me to the relevant specification?

                  Thanks,

                  Greg

                  On Fri, Jun 28, 2013 at 4:00 PM, Markus Lanthaler <markus.lanthaler@...> wrote:
                   

                  On Friday, June 28, 2013 2:10 PM, Greg Young wrote:


                  > On Fri, Jun 28, 2013 at 3:04 PM, Markus Lanthaler wrote:
                  >> If I would implement a client, I wouldn't keep polling the 404.
                  >> I would just assume your feed is broken.
                  >
                  >And if I returned a 204?

                  No, the status code that would make "most" sense IMO is 202.. but I would
                  still suggest to just hide the link till the resource gets created.

                  --
                  Markus Lanthaler
                  @markuslanthaler




                  --
                  Le doute n'est pas une condition agréable, mais la certitude est absurde.
                • Greg Young
                  Looks like our answer is to rename our links archive-prev/next as we intend them to be completely readable anyways. Thanks! Greg ... -- Le doute n est pas une
                  Message 8 of 23 , Jun 28, 2013
                  • 0 Attachment
                    Looks like our answer is to rename our links archive-prev/next as we intend them to be completely readable anyways.

                    Thanks!

                    Greg

                    On Fri, Jun 28, 2013 at 4:19 PM, Greg Young <gregoryyoung1@...> wrote:
                    btw in doing some research it would appear that RFC 5005 allows for either in the areas where reading of feeds is actually discussed.
                     
                    This process
                       should be repeated recursively until the client encounters a prev-
                       archive link relation that has been processed (the end of the archive
                       is indicated by a missing prev-archive link relation) or an error is
                       encountered.

                    prev/next reading is not discussed but would seem logical to work in the same way. I am unsure why it would be "broken" as you say to return an error. Can link me to the relevant specification?

                    Thanks,

                    Greg

                    On Fri, Jun 28, 2013 at 4:00 PM, Markus Lanthaler <markus.lanthaler@...> wrote:
                     

                    On Friday, June 28, 2013 2:10 PM, Greg Young wrote:


                    > On Fri, Jun 28, 2013 at 3:04 PM, Markus Lanthaler wrote:
                    >> If I would implement a client, I wouldn't keep polling the 404.
                    >> I would just assume your feed is broken.
                    >
                    >And if I returned a 204?

                    No, the status code that would make "most" sense IMO is 202.. but I would
                    still suggest to just hide the link till the resource gets created.

                    --
                    Markus Lanthaler
                    @markuslanthaler




                    --
                    Le doute n'est pas une condition agréable, mais la certitude est absurde.



                    --
                    Le doute n'est pas une condition agréable, mais la certitude est absurde.
                  • Markus Lanthaler
                    On Friday, June 28, 2013 3:19 PM, Greg Young wrote ... I didn t say that a spec prohibits it, how could it? You have to expect broken links everywhere. I just
                    Message 9 of 23 , Jun 28, 2013
                    • 0 Attachment
                      On Friday, June 28, 2013 3:19 PM, Greg Young wrote
                      > btw in doing some research it would appear that RFC 5005 allows
                      > for either in the areas where reading of feeds is actually discussed.

                      > This process
                      > should be repeated recursively until the client encounters a prev-
                      > archive link relation that has been processed (the end of the archive
                      > is indicated by a missing prev-archive link relation) or an error is
                      > encountered.
                      >
                      > prev/next reading is not discussed but would seem logical to work in the
                      > same way. I am unsure why it would be "broken" as you say to return an
                      > error. Can link me to the relevant specification?

                      I didn't say that a spec prohibits it, how could it? You have to expect
                      broken links everywhere. I just said I would assume that your implementation
                      is broken because it creates links that lead to a 404.

                      Errors shouldn't control the *normal* control flow, they should be
                      exceptions.



                      --
                      Markus Lanthaler
                      @markuslanthaler
                    • Greg Young
                      And doing it your way requires every uri we create to be put out twice (once as mutable and uncachable once as immutable and infinitely cachable). Take
                      Message 10 of 23 , Jun 28, 2013
                      • 0 Attachment
                        And doing it your way requires every uri we create to be put out twice (once as mutable and uncachable once as immutable and infinitely cachable).

                        Take archives. My current archive is therefor mutable. I have to rewrite the archive every time I add a new one. This sounds like a fairly complicated protocol as opposed to pre-writing what the next uri will be and giving some sort of status when it is not yet in existence. This becomes even more so if my archives are say monthly and widely distributed. This is especially true for us as our data is already immutable (prev/next were actually the wrong choice, we should be using archive)

                        Your client just has to say "more stuff will appear here" and don't move off my current document until it does (sounds a lot like a 204).

                        The server just puts out the archives as needed. No playing with etags or mutability (every uri is always infinitely cachable). 

                        The protocol makes even more sense when you consider dropping long polling on top of it (get me this thing that may or may not exist). 

                        Cheers,

                        Greg

                        On Fri, Jun 28, 2013 at 5:32 PM, Markus Lanthaler <markus.lanthaler@...> wrote:
                         

                        On Friday, June 28, 2013 3:19 PM, Greg Young wrote


                        > btw in doing some research it would appear that RFC 5005 allows
                        > for either in the areas where reading of feeds is actually discussed.

                        > This process
                        > should be repeated recursively until the client encounters a prev-
                        > archive link relation that has been processed (the end of the archive
                        > is indicated by a missing prev-archive link relation) or an error is
                        > encountered.
                        >
                        > prev/next reading is not discussed but would seem logical to work in the
                        > same way. I am unsure why it would be "broken" as you say to return an
                        > error. Can link me to the relevant specification?

                        I didn't say that a spec prohibits it, how could it? You have to expect
                        broken links everywhere. I just said I would assume that your implementation
                        is broken because it creates links that lead to a 404.

                        Errors shouldn't control the *normal* control flow, they should be
                        exceptions.

                        --
                        Markus Lanthaler
                        @markuslanthaler




                        --
                        Le doute n'est pas une condition agréable, mais la certitude est absurde.
                      • Greg Young
                        btw just to add to this. The specification contradicts itself. If I have to adjust a prev link on the archive ... How can: The requirement that archive
                        Message 11 of 23 , Jun 28, 2013
                        • 0 Attachment
                          btw just to add to this. The specification contradicts itself. If I
                          have to adjust a prev link on the archive ...

                          How can:

                          " The requirement that archive documents be stable allows clients to

                          safely assume that if they have retrieved one in the past, it will
                          not meaningfully change in the future. "

                          I would tend to read this as immutability.

                          Is changing a prev link not a meaningful change? I would consider it
                          very meaningful.

                          Cheers,

                          Greg


                          On Fri, Jun 28, 2013 at 5:45 PM, Greg Young <gregoryyoung1@...> wrote:
                          >
                          > And doing it your way requires every uri we create to be put out twice (once as mutable and uncachable once as immutable and infinitely cachable).
                          >
                          > Take archives. My current archive is therefor mutable. I have to rewrite the archive every time I add a new one. This sounds like a fairly complicated protocol as opposed to pre-writing what the next uri will be and giving some sort of status when it is not yet in existence. This becomes even more so if my archives are say monthly and widely distributed. This is especially true for us as our data is already immutable (prev/next were actually the wrong choice, we should be using archive)
                          >
                          > Your client just has to say "more stuff will appear here" and don't move off my current document until it does (sounds a lot like a 204).
                          >
                          > The server just puts out the archives as needed. No playing with etags or mutability (every uri is always infinitely cachable).
                          >
                          > The protocol makes even more sense when you consider dropping long polling on top of it (get me this thing that may or may not exist).
                          >
                          > Cheers,
                          >
                          > Greg
                          >
                          > On Fri, Jun 28, 2013 at 5:32 PM, Markus Lanthaler <markus.lanthaler@...> wrote:
                          >>
                          >>
                          >>
                          >> On Friday, June 28, 2013 3:19 PM, Greg Young wrote
                          >>
                          >>
                          >> > btw in doing some research it would appear that RFC 5005 allows
                          >> > for either in the areas where reading of feeds is actually discussed.
                          >> >
                          >> > This process
                          >> > should be repeated recursively until the client encounters a prev-
                          >> > archive link relation that has been processed (the end of the archive
                          >> > is indicated by a missing prev-archive link relation) or an error is
                          >> > encountered.
                          >> >
                          >> > prev/next reading is not discussed but would seem logical to work in the
                          >> > same way. I am unsure why it would be "broken" as you say to return an
                          >> > error. Can link me to the relevant specification?
                          >>
                          >> I didn't say that a spec prohibits it, how could it? You have to expect
                          >> broken links everywhere. I just said I would assume that your implementation
                          >> is broken because it creates links that lead to a 404.
                          >>
                          >> Errors shouldn't control the *normal* control flow, they should be
                          >> exceptions.
                          >>
                          >> --
                          >> Markus Lanthaler
                          >> @markuslanthaler
                          >>
                          >>
                          >
                          >
                          >
                          >
                          > --
                          > Le doute n'est pas une condition agréable, mais la certitude est absurde.




                          --
                          Le doute n'est pas une condition agréable, mais la certitude est absurde.
                        • Erik Wilde
                          could a link hint help? saying this will eventually work but don t expect it to work before ... ? cheers, dret. ... could a link hint help? saying this will
                          Message 12 of 23 , Jun 28, 2013
                          • 0 Attachment
                            could a link hint help? saying "this will eventually work but don't expect it to work before ..."? cheers, dret.

                            On Jun 28, 2013, at 7:45, Greg Young <gregoryyoung1@...> wrote:

                            And doing it your way requires every uri we create to be put out twice (once as mutable and uncachable once as immutable and infinitely cachable).

                            Take archives. My current archive is therefor mutable. I have to rewrite the archive every time I add a new one. This sounds like a fairly complicated protocol as opposed to pre-writing what the next uri will be and giving some sort of status when it is not yet in existence. This becomes even more so if my archives are say monthly and widely distributed. This is especially true for us as our data is already immutable (prev/next were actually the wrong choice, we should be using archive)

                            Your client just has to say "more stuff will appear here" and don't move off my current document until it does (sounds a lot like a 204).

                            The server just puts out the archives as needed. No playing with etags or mutability (every uri is always infinitely cachable). 

                            The protocol makes even more sense when you consider dropping long polling on top of it (get me this thing that may or may not exist). 

                            Cheers,

                            Greg

                            On Fri, Jun 28, 2013 at 5:32 PM, Markus Lanthaler <markus.lanthaler@...> wrote:
                             

                            On Friday, June 28, 2013 3:19 PM, Greg Young wrote


                            > btw in doing some research it would appear that RFC 5005 allows
                            > for either in the areas where reading of feeds is actually discussed.

                            > This process
                            > should be repeated recursively until the client encounters a prev-
                            > archive link relation that has been processed (the end of the archive
                            > is indicated by a missing prev-archive link relation) or an error is
                            > encountered.
                            >
                            > prev/next reading is not discussed but would seem logical to work in the
                            > same way. I am unsure why it would be "broken" as you say to return an
                            > error. Can link me to the relevant specification?

                            I didn't say that a spec prohibits it, how could it? You have to expect
                            broken links everywhere. I just said I would assume that your implementation
                            is broken because it creates links that lead to a 404.

                            Errors shouldn't control the *normal* control flow, they should be
                            exceptions.

                            --
                            Markus Lanthaler
                            @markuslanthaler




                            --
                            Le doute n'est pas une condition agréable, mais la certitude est absurde.
                          • Greg Young
                            we dont know when it will work. ... -- Le doute n est pas une condition agréable, mais la certitude est absurde. we dont know when it will work. On Fri, Jun
                            Message 13 of 23 , Jun 28, 2013
                            • 0 Attachment
                              we dont know when it will work.

                              On Fri, Jun 28, 2013 at 6:14 PM, Erik Wilde <dret@...> wrote:
                              could a link hint help? saying "this will eventually work but don't expect it to work before ..."? cheers, dret.

                              On Jun 28, 2013, at 7:45, Greg Young <gregoryyoung1@...> wrote:



                              And doing it your way requires every uri we create to be put out twice (once as mutable and uncachable once as immutable and infinitely cachable).

                              Take archives. My current archive is therefor mutable. I have to rewrite the archive every time I add a new one. This sounds like a fairly complicated protocol as opposed to pre-writing what the next uri will be and giving some sort of status when it is not yet in existence. This becomes even more so if my archives are say monthly and widely distributed. This is especially true for us as our data is already immutable (prev/next were actually the wrong choice, we should be using archive)

                              Your client just has to say "more stuff will appear here" and don't move off my current document until it does (sounds a lot like a 204).

                              The server just puts out the archives as needed. No playing with etags or mutability (every uri is always infinitely cachable). 

                              The protocol makes even more sense when you consider dropping long polling on top of it (get me this thing that may or may not exist). 

                              Cheers,

                              Greg

                              On Fri, Jun 28, 2013 at 5:32 PM, Markus Lanthaler <markus.lanthaler@...> wrote:
                               

                              On Friday, June 28, 2013 3:19 PM, Greg Young wrote


                              > btw in doing some research it would appear that RFC 5005 allows
                              > for either in the areas where reading of feeds is actually discussed.

                              > This process
                              > should be repeated recursively until the client encounters a prev-
                              > archive link relation that has been processed (the end of the archive
                              > is indicated by a missing prev-archive link relation) or an error is
                              > encountered.
                              >
                              > prev/next reading is not discussed but would seem logical to work in the
                              > same way. I am unsure why it would be "broken" as you say to return an
                              > error. Can link me to the relevant specification?

                              I didn't say that a spec prohibits it, how could it? You have to expect
                              broken links everywhere. I just said I would assume that your implementation
                              is broken because it creates links that lead to a 404.

                              Errors shouldn't control the *normal* control flow, they should be
                              exceptions.

                              --
                              Markus Lanthaler
                              @markuslanthaler




                              --
                              Le doute n'est pas une condition agréable, mais la certitude est absurde.





                              --
                              Le doute n'est pas une condition agréable, mais la certitude est absurde.
                            • Erik Wilde
                              i guess then there s little to communicate in a link hint (other than a very vague maybe don t try before X ). but if the links are expected to work at some
                              Message 14 of 23 , Jun 28, 2013
                              • 0 Attachment
                                i guess then there's little to communicate in a link hint (other than a
                                very vague "maybe don't try before X"). but if the links are expected to
                                work at some predictable time (a daily newsletter, for example), then i
                                guess you could you a link hint to inform clients about this.

                                On 2013-06-28 08:23 , Greg Young wrote:
                                > we dont know when it will work.
                                >
                                > On Fri, Jun 28, 2013 at 6:14 PM, Erik Wilde <dret@...
                                > <mailto:dret@...>> wrote:
                                >
                                > could a link hint help? saying "this will eventually work but don't
                                > expect it to work before ..."? cheers, dret.
                                >
                                > On Jun 28, 2013, at 7:45, Greg Young <gregoryyoung1@...
                                > <mailto:gregoryyoung1@...>> wrote:
                                >
                                >>
                                >>
                                >> And doing it your way requires every uri we create to be put out
                                >> twice (once as mutable and uncachable once as immutable and
                                >> infinitely cachable).
                                >>
                                >> Take archives. My current archive is therefor mutable. I have to
                                >> rewrite the archive every time I add a new one. This sounds like a
                                >> fairly complicated protocol as opposed to pre-writing what the
                                >> next uri will be and giving some sort of status when it is not yet
                                >> in existence. This becomes even more so if my archives are say
                                >> monthly and widely distributed. This is especially true for us as
                                >> our data is already immutable (prev/next were actually the wrong
                                >> choice, we should be using archive)
                                >>
                                >> Your client just has to say "more stuff will appear here" and
                                >> don't move off my current document until it does (sounds a lot
                                >> like a 204).
                                >>
                                >> The server just puts out the archives as needed. No playing with
                                >> etags or mutability (every uri is always infinitely cachable).
                                >>
                                >> The protocol makes even more sense when you consider dropping long
                                >> polling on top of it (get me this thing that may or may not exist).
                                >>
                                >> Cheers,
                                >>
                                >> Greg
                                >>
                                >> On Fri, Jun 28, 2013 at 5:32 PM, Markus Lanthaler
                                >> <markus.lanthaler@... <mailto:markus.lanthaler@...>> wrote:
                                >>
                                >> __
                                >>
                                >> On Friday, June 28, 2013 3:19 PM, Greg Young wrote
                                >>
                                >>
                                >> > btw in doing some research it would appear that RFC 5005 allows
                                >> > for either in the areas where reading of feeds is actually
                                >> discussed.
                                >> >
                                >> > This process
                                >> > should be repeated recursively until the client encounters a
                                >> prev-
                                >> > archive link relation that has been processed (the end of
                                >> the archive
                                >> > is indicated by a missing prev-archive link relation) or an
                                >> error is
                                >> > encountered.
                                >> >
                                >> > prev/next reading is not discussed but would seem logical to
                                >> work in the
                                >> > same way. I am unsure why it would be "broken" as you say to
                                >> return an
                                >> > error. Can link me to the relevant specification?
                                >>
                                >> I didn't say that a spec prohibits it, how could it? You have
                                >> to expect
                                >> broken links everywhere. I just said I would assume that your
                                >> implementation
                                >> is broken because it creates links that lead to a 404.
                                >>
                                >> Errors shouldn't control the *normal* control flow, they should be
                                >> exceptions.
                                >>
                                >> --
                                >> Markus Lanthaler
                                >> @markuslanthaler
                                >>
                                >>
                                >>
                                >>
                                >> --
                                >> Le doute n'est pas une condition agréable, mais la certitude est
                                >> absurde.
                                >>
                                >>
                                >>
                                >
                                >
                                >
                                >
                                > --
                                > Le doute n'est pas une condition agréable, mais la certitude est absurde.

                                --
                                erik wilde | mailto:dret@... - tel:+1-510-2061079 |
                                | UC Berkeley - School of Information (ISchool) |
                                | http://dret.net/netdret http://twitter.com/dret |
                              • Greg Young
                                not predictable (its an event stream) it could be 1ms or 30 days. we have switched to archive-prev (our stream is complete and non-lossy) but it really feels
                                Message 15 of 23 , Jun 28, 2013
                                • 0 Attachment
                                  not predictable (its an event stream) it could be 1ms or 30 days.

                                  we have switched to archive-prev (our stream is complete and
                                  non-lossy) but it really feels like an immutable prev/next is needed
                                  to be standardized.

                                  we figured a 204 "don't change your document" + retry fit the bill.
                                  maybe not ...

                                  As of now there is no good way to do what we do (non-lossy stream)
                                  under 5005. As such we will have to just do something I guess and then
                                  try to get a spec change. The changes are not huge and I think are in
                                  the spirit of the spec (weird how they want mutability and
                                  immutability at the same time).

                                  Greg

                                  On Fri, Jun 28, 2013 at 6:44 PM, Erik Wilde <dret@...> wrote:
                                  > i guess then there's little to communicate in a link hint (other than a very
                                  > vague "maybe don't try before X"). but if the links are expected to work at
                                  > some predictable time (a daily newsletter, for example), then i guess you
                                  > could you a link hint to inform clients about this.
                                  >
                                  >
                                  > On 2013-06-28 08:23 , Greg Young wrote:
                                  >>
                                  >> we dont know when it will work.
                                  >>
                                  >> On Fri, Jun 28, 2013 at 6:14 PM, Erik Wilde <dret@...
                                  >> <mailto:dret@...>> wrote:
                                  >>
                                  >> could a link hint help? saying "this will eventually work but don't
                                  >> expect it to work before ..."? cheers, dret.
                                  >>
                                  >> On Jun 28, 2013, at 7:45, Greg Young <gregoryyoung1@...
                                  >> <mailto:gregoryyoung1@...>> wrote:
                                  >>
                                  >>>
                                  >>>
                                  >>> And doing it your way requires every uri we create to be put out
                                  >>> twice (once as mutable and uncachable once as immutable and
                                  >>> infinitely cachable).
                                  >>>
                                  >>> Take archives. My current archive is therefor mutable. I have to
                                  >>> rewrite the archive every time I add a new one. This sounds like a
                                  >>> fairly complicated protocol as opposed to pre-writing what the
                                  >>> next uri will be and giving some sort of status when it is not yet
                                  >>> in existence. This becomes even more so if my archives are say
                                  >>> monthly and widely distributed. This is especially true for us as
                                  >>> our data is already immutable (prev/next were actually the wrong
                                  >>> choice, we should be using archive)
                                  >>>
                                  >>> Your client just has to say "more stuff will appear here" and
                                  >>> don't move off my current document until it does (sounds a lot
                                  >>> like a 204).
                                  >>>
                                  >>> The server just puts out the archives as needed. No playing with
                                  >>> etags or mutability (every uri is always infinitely cachable).
                                  >>>
                                  >>> The protocol makes even more sense when you consider dropping long
                                  >>> polling on top of it (get me this thing that may or may not exist).
                                  >>>
                                  >>> Cheers,
                                  >>>
                                  >>> Greg
                                  >>>
                                  >>> On Fri, Jun 28, 2013 at 5:32 PM, Markus Lanthaler
                                  >>> <markus.lanthaler@... <mailto:markus.lanthaler@...>> wrote:
                                  >>>
                                  >>> __
                                  >>>
                                  >>>
                                  >>> On Friday, June 28, 2013 3:19 PM, Greg Young wrote
                                  >>>
                                  >>>
                                  >>> > btw in doing some research it would appear that RFC 5005 allows
                                  >>> > for either in the areas where reading of feeds is actually
                                  >>> discussed.
                                  >>> >
                                  >>> > This process
                                  >>> > should be repeated recursively until the client encounters a
                                  >>> prev-
                                  >>> > archive link relation that has been processed (the end of
                                  >>> the archive
                                  >>> > is indicated by a missing prev-archive link relation) or an
                                  >>> error is
                                  >>> > encountered.
                                  >>> >
                                  >>> > prev/next reading is not discussed but would seem logical to
                                  >>> work in the
                                  >>> > same way. I am unsure why it would be "broken" as you say to
                                  >>> return an
                                  >>> > error. Can link me to the relevant specification?
                                  >>>
                                  >>> I didn't say that a spec prohibits it, how could it? You have
                                  >>> to expect
                                  >>> broken links everywhere. I just said I would assume that your
                                  >>> implementation
                                  >>> is broken because it creates links that lead to a 404.
                                  >>>
                                  >>> Errors shouldn't control the *normal* control flow, they should
                                  >>> be
                                  >>> exceptions.
                                  >>>
                                  >>> --
                                  >>> Markus Lanthaler
                                  >>> @markuslanthaler
                                  >>>
                                  >>>
                                  >>>
                                  >>>
                                  >>> --
                                  >>> Le doute n'est pas une condition agréable, mais la certitude est
                                  >>> absurde.
                                  >>>
                                  >>>
                                  >>>
                                  >>
                                  >>
                                  >>
                                  >>
                                  >>
                                  >> --
                                  >> Le doute n'est pas une condition agréable, mais la certitude est absurde.
                                  >
                                  >
                                  > --
                                  > erik wilde | mailto:dret@... - tel:+1-510-2061079 |
                                  > | UC Berkeley - School of Information (ISchool) |
                                  > | http://dret.net/netdret http://twitter.com/dret |



                                  --
                                  Le doute n'est pas une condition agréable, mais la certitude est absurde.
                                • Erik Mogensen
                                  ... also use the prev link of the first page of a feed to link to the page containing the items that will be . We find that it s an extremely resilient way
                                  Message 16 of 23 , Jun 28, 2013
                                  • 0 Attachment

                                    On Fri, Jun 28, 2013 at 12:16 PM, Greg Young <gregoryyoung1@...> wrote:
                                     

                                    We have a mature atom implementation for distribution of event streams
                                    and as always the devil is in the details...

                                    The issue revolves around prev links. At some point on an atom feed
                                    you will hit the current item, prev will point to the next item that
                                    does not yet exist. Most implementations I can find 404 this uri and
                                    the client has the logic to keep polling the 404. This seems off to
                                    me, I would expect this to be a 204. Then if I change my uri structure
                                    it becomes a 404, otherwise it would seem that if I change my uri
                                    structure all prev uris would need to redirect to the new ones (which
                                    would be a rather large undertaking).

                                    I couldn't find any details on this in relevant specs, has anyone seen this?

                                    Greg, I'd just like to explain how we do more or less the same thing.  We also use the prev link of the "first page" of a feed to link to the page containing the items that "will be".  We find that it's an extremely resilient way of dissipating information about things that fit in atom feeds, and a relatively efficient system too.

                                    Infinitely cacheable resources are nice, but instead of 404, or 204 and so on, we expose an empty feed document, with a next link but no prev link.  When things happen, the page (when refreshed) contains one or more entries, and becomes cacheable.

                                    Our "head" document is very small; contains no real data so it's easy to generate; supports cache validation, and if it's incorrectly cached as fresh by an intermediary or a caching client, it only impedes the ability of clients to advance to the "newer" entries.  Eventually, they will advance to the next state. 

                                    If I were to do this again, I might consider using 302 some other temporary redirect, instead of the 404 or the empty feed; redirect back to the current (infinitely cacheable) first page.  When a new item appears, the 302 would turn into the "new" first page" a 200 with new next/prev links, 

                                    I wonder if "Cache-Control" or "Retry-After" would be useful for 302 responses, to ask clients to back off under heavy load.
                                    --
                                    -mogsie-
                                  • Greg Young
                                    We have considered this empty feed page as well! We were thinking its slightly more efficient (and understandable) to have a status code but maybe not
                                    Message 17 of 23 , Jun 28, 2013
                                    • 0 Attachment
                                      We have considered this empty feed page as well! We were thinking its
                                      slightly more efficient (and understandable) to have a status code but
                                      maybe not (thinking client logic).

                                      If I were to do this again, I might consider using 302 some other
                                      temporary redirect, instead of the 404 or the empty feed; redirect
                                      back to the current (infinitely cacheable) first page. When a new
                                      item appears, the 302 would turn into the "new" first page" a 200 with
                                      new next/prev links,

                                      I am curious why not a 204. It would seem to be a reasonable fit here
                                      "keep your doc, retry after ..."

                                      Cheers,

                                      Greg

                                      On Fri, Jun 28, 2013 at 7:09 PM, Erik Mogensen <erik@...> wrote:
                                      >
                                      > On Fri, Jun 28, 2013 at 12:16 PM, Greg Young <gregoryyoung1@...>
                                      > wrote:
                                      >>
                                      >>
                                      >>
                                      >> We have a mature atom implementation for distribution of event streams
                                      >> and as always the devil is in the details...
                                      >>
                                      >> The issue revolves around prev links. At some point on an atom feed
                                      >> you will hit the current item, prev will point to the next item that
                                      >> does not yet exist. Most implementations I can find 404 this uri and
                                      >> the client has the logic to keep polling the 404. This seems off to
                                      >> me, I would expect this to be a 204. Then if I change my uri structure
                                      >> it becomes a 404, otherwise it would seem that if I change my uri
                                      >> structure all prev uris would need to redirect to the new ones (which
                                      >> would be a rather large undertaking).
                                      >>
                                      >> I couldn't find any details on this in relevant specs, has anyone seen
                                      >> this?
                                      >>
                                      > Greg, I'd just like to explain how we do more or less the same thing. We
                                      > also use the prev link of the "first page" of a feed to link to the page
                                      > containing the items that "will be". We find that it's an extremely
                                      > resilient way of dissipating information about things that fit in atom
                                      > feeds, and a relatively efficient system too.
                                      >
                                      > Infinitely cacheable resources are nice, but instead of 404, or 204 and so
                                      > on, we expose an empty feed document, with a next link but no prev link.
                                      > When things happen, the page (when refreshed) contains one or more entries,
                                      > and becomes cacheable.
                                      >
                                      > Our "head" document is very small; contains no real data so it's easy to
                                      > generate; supports cache validation, and if it's incorrectly cached as fresh
                                      > by an intermediary or a caching client, it only impedes the ability of
                                      > clients to advance to the "newer" entries. Eventually, they will advance to
                                      > the next state.
                                      >
                                      > If I were to do this again, I might consider using 302 some other temporary
                                      > redirect, instead of the 404 or the empty feed; redirect back to the current
                                      > (infinitely cacheable) first page. When a new item appears, the 302 would
                                      > turn into the "new" first page" a 200 with new next/prev links,
                                      >
                                      > I wonder if "Cache-Control" or "Retry-After" would be useful for 302
                                      > responses, to ask clients to back off under heavy load.
                                      > --
                                      > -mogsie-



                                      --
                                      Le doute n'est pas une condition agréable, mais la certitude est absurde.
                                    • Erik Mogensen
                                      ... We were probably thinking that the empty page would fit nicely within an IP packet anyway, and could easily support ETag validation, and provide
                                      Message 18 of 23 , Jun 28, 2013
                                      • 0 Attachment
                                        On Fri, Jun 28, 2013 at 6:14 PM, Greg Young <gregoryyoung1@...> wrote:
                                         

                                        We have considered this empty feed page as well! We were thinking its
                                        slightly more efficient (and understandable) to have a status code but
                                        maybe not (thinking client logic).

                                        We were probably thinking that the empty page would fit nicely within an IP packet anyway, and could easily support ETag validation, and provide information about the current state "there's nothing to process now".

                                        I am curious why not a 204. It would seem to be a reasonable fit here
                                        "keep your doc, retry after ..."


                                        Hmm. after reading up on RFC-2616 I'd agree with you.  My understanding of 204 means that you should "stay where you were" and then I guess the client would eventually retry.

                                        RFC2616, section 10.2.5 "204 NO CONTENT" says: "If the client is a user agent, it SHOULD NOT change its document view from that which caused the request to be sent" — This should cause the same or a very similar outcome, I guess.

                                        I guess we should update our client to react accordingly when we encounter a 204. :-)
                                        --
                                        -mogsie-
                                      • Ben Niven-Jenkins
                                        ... Why not use feed paging and use prev-archive to indicate the previous page and keep your main feed as the current page. Client just polls the current
                                        Message 19 of 23 , Jun 28, 2013
                                        • 0 Attachment
                                          On 28 Jun 2013, at 11:16, Greg Young <gregoryyoung1@...> wrote:

                                          > We have a mature atom implementation for distribution of event streams
                                          > and as always the devil is in the details...
                                          >
                                          > The issue revolves around prev links. At some point on an atom feed
                                          > you will hit the current item, prev will point to the next item that
                                          > does not yet exist.

                                          Why not use feed paging and use prev-archive to indicate the previous page and keep your "main" feed as the current page.

                                          Client just polls the current page and if it needs to go further back in time it follows prev-archive links until it finds what it wants.

                                          You then avoid the problem of links to things that don't exist or updating already archived pages.

                                          Ben
                                        • Greg Young
                                          Our whole feed is nonlossy and immutable. ... -- Le doute n est pas une condition agréable, mais la certitude est absurde. Our whole feed is nonlossy and
                                          Message 20 of 23 , Jun 28, 2013
                                          • 0 Attachment
                                            Our whole feed is nonlossy and immutable.

                                            On Friday, June 28, 2013, Ben Niven-Jenkins wrote:


                                            On 28 Jun 2013, at 11:16, Greg Young <gregoryyoung1@...> wrote:

                                            > We have a mature atom implementation for distribution of event streams
                                            > and as always the devil is in the details...
                                            >
                                            > The issue revolves around prev links. At some point on an atom feed
                                            > you will hit the current item, prev will point to the next item that
                                            > does not yet exist.

                                            Why not use feed paging and use prev-archive to indicate the previous page and keep your "main" feed as the current page.

                                            Client just polls the current page and if it needs to go further back in time it follows prev-archive links until it finds what it wants.

                                            You then avoid the problem of links to things that don't exist or updating already archived pages.

                                            Ben


                                            --
                                            Le doute n'est pas une condition agréable, mais la certitude est absurde.
                                          • Ben Niven-Jenkins
                                            ... Can you expand on what you mean by this. Feed archiving doesn t lose information and doesn t require changes to a page once it is first generated. Or do
                                            Message 21 of 23 , Jun 28, 2013
                                            • 0 Attachment


                                              On 28 Jun 2013, at 18:32, Greg Young <gregoryyoung1@...> wrote:

                                              Our whole feed is nonlossy and immutable.

                                              Can you expand on what you mean by this. Feed archiving doesn't lose information and doesn't require changes to a page once it is first generated. 

                                              Or do you mean the client cannot handle the content of the URI that represents the entry point into the feed changing and expects the content of that URI to stay always constant so the clients needs to page "forward in time" to find the latest entries?

                                              Ben

                                              On Friday, June 28, 2013, Ben Niven-Jenkins wrote:


                                              On 28 Jun 2013, at 11:16, Greg Young <gregoryyoung1@...> wrote:

                                              > We have a mature atom implementation for distribution of event streams
                                              > and as always the devil is in the details...
                                              >
                                              > The issue revolves around prev links. At some point on an atom feed
                                              > you will hit the current item, prev will point to the next item that
                                              > does not yet exist.

                                              Why not use feed paging and use prev-archive to indicate the previous page and keep your "main" feed as the current page.

                                              Client just polls the current page and if it needs to go further back in time it follows prev-archive links until it finds what it wants.

                                              You then avoid the problem of links to things that don't exist or updating already archived pages.

                                              Ben


                                              --
                                              Le doute n'est pas une condition agréable, mais la certitude est absurde.
                                            • Greg Young
                                              We could do all over prev next even with no archiving. Every uri we create is immutable. Spec defines additional logic needed for prev/next which is unneeded
                                              Message 22 of 23 , Jun 28, 2013
                                              • 0 Attachment
                                                We could do all over prev next even with no archiving. Every uri we create is immutable. Spec defines additional logic needed for prev/next which is unneeded for us so archive gives us a simpler client API (also explicit that we are non lossy)

                                                On Friday, June 28, 2013, Ben Niven-Jenkins wrote:


                                                On 28 Jun 2013, at 18:32, Greg Young <gregoryyoung1@...> wrote:

                                                Our whole feed is nonlossy and immutable.

                                                Can you expand on what you mean by this. Feed archiving doesn't lose information and doesn't require changes to a page once it is first generated. 

                                                Or do you mean the client cannot handle the content of the URI that represents the entry point into the feed changing and expects the content of that URI to stay always constant so the clients needs to page "forward in time" to find the latest entries?

                                                Ben

                                                On Friday, June 28, 2013, Ben Niven-Jenkins wrote:


                                                On 28 Jun 2013, at 11:16, Greg Young <gregoryyoung1@...> wrote:

                                                > We have a mature atom implementation for distribution of event streams
                                                > and as always the devil is in the details...
                                                >
                                                > The issue revolves around prev links. At some point on an atom feed
                                                > you will hit the current item, prev will point to the next item that
                                                > does not yet exist.

                                                Why not use feed paging and use prev-archive to indicate the previous page and keep your "main" feed as the current page.

                                                Client just polls the current page and if it needs to go further back in time it follows prev-archive links until it finds what it wants.

                                                You then avoid the problem of links to things that don't exist or updating already archived pages.

                                                Ben


                                                --
                                                Le doute n'est pas une condition agréable, mais la certitude est absurde.


                                                --
                                                Le doute n'est pas une condition agréable, mais la certitude est absurde.
                                              • Greg Young
                                                To be clear we want a cross of prev and archive prev as they were obviously designed for mutable feeds. ... -- Le doute n est pas une condition agréable, mais
                                                Message 23 of 23 , Jun 28, 2013
                                                • 0 Attachment
                                                  To be clear we want a cross of prev and archive prev as they were obviously designed for mutable feeds.

                                                  On Friday, June 28, 2013, Greg Young wrote:
                                                  We could do all over prev next even with no archiving. Every uri we create is immutable. Spec defines additional logic needed for prev/next which is unneeded for us so archive gives us a simpler client API (also explicit that we are non lossy)

                                                  On Friday, June 28, 2013, Ben Niven-Jenkins wrote:


                                                  On 28 Jun 2013, at 18:32, Greg Young <gregoryyoung1@...> wrote:

                                                  Our whole feed is nonlossy and immutable.

                                                  Can you expand on what you mean by this. Feed archiving doesn't lose information and doesn't require changes to a page once it is first generated. 

                                                  Or do you mean the client cannot handle the content of the URI that represents the entry point into the feed changing and expects the content of that URI to stay always constant so the clients needs to page "forward in time" to find the latest entries?

                                                  Ben

                                                  On Friday, June 28, 2013, Ben Niven-Jenkins wrote:


                                                  On 28 Jun 2013, at 11:16, Greg Young <gregoryyoung1@...> wrote:

                                                  > We have a mature atom implementation for distribution of event streams
                                                  > and as always the devil is in the details...
                                                  >
                                                  > The issue revolves around prev links. At some point on an atom feed
                                                  > you will hit the current item, prev will point to the next item that
                                                  > does not yet exist.

                                                  Why not use feed paging and use prev-archive to indicate the previous page and keep your "main" feed as the current page.

                                                  Client just polls the current page and if it needs to go further back in time it follows prev-archive links until it finds what it wants.

                                                  You then avoid the problem of links to things that don't exist or updating already archived pages.

                                                  Ben


                                                  --
                                                  Le doute n'est pas une condition agréable, mais la certitude est absurde.


                                                  --
                                                  Le doute n'est pas une condition agréable, mais la certitude est absurde.


                                                  --
                                                  Le doute n'est pas une condition agréable, mais la certitude est absurde.
                                                Your message has been successfully submitted and would be delivered to recipients shortly.