Loading ...
Sorry, an error occurred while loading the content.

Updating metadata of multiple resources

Expand Messages
  • Stefano Masini
    Hi, I m still trying to figure rest out. Let s see if I hit a sweet spot or if I m just missing something (more likely, please forgive me). I have several
    Message 1 of 19 , May 2, 2008
    • 0 Attachment
      Hi,

      I'm still trying to figure rest out. Let's see if I hit a sweet spot
      or if I'm just missing something (more likely, please forgive me).

      I have several images, each with its own uri that have metadata.
      If I want to update the metadata of a given image I send a POST
      request on its uri. Right?
      What if I want to atomically update some metadata (say, a tag) for a
      group of images, how do I approach that?
      Sending multiple POST requests, one for each uri, is not atomic, so I
      can't accept it.

      I thought about posting on the common parent of all the image uris
      (say, if I have /images/1, /images/2, etc., then I post on /images).
      In the parameters of the post request I would put a list of all the
      imaged ids (or entire uri), so that the server can handle the request
      atomically.

      But then what should I get in return? What should /images return if I
      issued a GET? The whole repository? Naaa... too big.
      Isn't the POST supposed to return the same representation? Just after
      handling the posted request.

      How much am I missing from the rest picture?

      Thanks
      Stefano
    • Gary Bernhardt
      On Fri, May 2, 2008 at 4:56 PM, Stefano Masini ... First, it sounds like you want PUT, not POST. PUT means create or replace the entity ; POST means accept
      Message 2 of 19 , May 2, 2008
      • 0 Attachment
        On Fri, May 2, 2008 at 4:56 PM, Stefano Masini
        <stefano.masini@...> wrote:
        > Hi,
        >
        > I'm still trying to figure rest out. Let's see if I hit a sweet spot
        > or if I'm just missing something (more likely, please forgive me).
        >
        > I have several images, each with its own uri that have metadata.
        > If I want to update the metadata of a given image I send a POST
        > request on its uri. Right?
        > What if I want to atomically update some metadata (say, a tag) for a
        > group of images, how do I approach that?
        > Sending multiple POST requests, one for each uri, is not atomic, so I
        > can't accept it.

        First, it sounds like you want PUT, not POST. PUT means "create or
        replace the entity"; POST means "accept this entity as a child". What
        you want to do here is GET the full representation, make whatever
        changes you want, then PUT the whole thing back. It's a
        "read-modify-write" operation.

        > I thought about posting on the common parent of all the image uris
        > (say, if I have /images/1, /images/2, etc., then I post on /images).
        > In the parameters of the post request I would put a list of all the
        > imaged ids (or entire uri), so that the server can handle the request
        > atomically.

        I'd define a way to refer to a specific set of images. Say you want
        to modify images 1, 4, and 13. You can PUT (and GET) to
        "/images/1,4,13" with an entity fully describing three images. This
        is just like your proposed idea, except you're referring to a limited
        set of images instead of all of them. The server then atomically
        replaces images 1, 4, and 13 with the entities you provided.

        One thing to note is that URIs like this require more complex linking.
        The client needs to be able to construct those "1,4,13" URIs somehow,
        but it must be told how to, at runtime, by the server; otherwise it
        would violate HATEoAS. You need a way to say something like <a
        href="/images/\d(,\d)*">, where the href there is actually a regex
        describing how to build a URI out of a series of integers. Maybe
        someone else can chime on how to do this well, because I have no idea.
        I do know that you do *not* want to actually use regexes, though. :)

        > But then what should I get in return? What should /images return if I
        > issued a GET? The whole repository? Naaa... too big.
        > Isn't the POST supposed to return the same representation? Just after
        > handling the posted request.

        This question is answered by the limited-subset solution above. When
        you GET "/images/1,4,13", you get back exactly what you'd expect -
        images 1, 4, and 13, just as they were when you PUT them in.

        > How much am I missing from the rest picture?

        Well, the PUT/POST thing, but everyone makes that mistake at first. :)
        I think another thing people often miss is that the URI space doesn't
        have to be rigid or static. That's exactly the issue here. What
        you're trying to do is modify a subset of a collection, so just say it
        in the URI! :)

        --
        Gary
        http://blog.extracheese.org
      • Leonardo Boiko
        For the case of image tags (like flickr, etc.) maybe you could expose the tags as resources. Then if you want to add the tag chunkybacon to a set of images,
        Message 3 of 19 , May 2, 2008
        • 0 Attachment
          For the case of image tags (like flickr, etc.) maybe you could expose
          the tags as resources. Then if you want to add the tag "chunkybacon"
          to a set of images, you could POST to e.g. /tag/chunkybacon/ an
          entity-body with the list of the URIs of images you want to add the
          tag. Of course if you GET /tag/chunkybacon/ you'll see a
          representation of a list of all chunkybacon images so far.

          A similar trick using lists is to expose a list of all image tags
          under /images/<id>/tags; then you can POST multiple tags to a single
          image. A more complex application would be to support image sets at
          URI level, like "/images/<id1>;<id2>;<id3>"; then you can POST
          multiple tags to multiple images at "/images/<id1>;<idN>/tags"... You
          got the idea. Whatever method you use, be sure to link to or create
          forms to it elsewhere.

          Side note: the "list URIs" above, as a not-much-known convention, use
          "," as separator when order matters and ";" when it doesn't. Note also
          that URIs have a maximum lenght.



          On 5/2/08, Gary Bernhardt <gary.bernhardt@...> wrote:
          > On Fri, May 2, 2008 at 4:56 PM, Stefano Masini
          > <stefano.masini@...> wrote:
          > > Hi,
          > >
          > > I'm still trying to figure rest out. Let's see if I hit a sweet spot
          > > or if I'm just missing something (more likely, please forgive me).
          > >
          > > I have several images, each with its own uri that have metadata.
          > > If I want to update the metadata of a given image I send a POST
          > > request on its uri. Right?
          > > What if I want to atomically update some metadata (say, a tag) for a
          > > group of images, how do I approach that?
          > > Sending multiple POST requests, one for each uri, is not atomic, so I
          > > can't accept it.
          >
          > First, it sounds like you want PUT, not POST. PUT means "create or
          > replace the entity"; POST means "accept this entity as a child". What
          > you want to do here is GET the full representation, make whatever
          > changes you want, then PUT the whole thing back. It's a
          > "read-modify-write" operation.
          >
          > > I thought about posting on the common parent of all the image uris
          > > (say, if I have /images/1, /images/2, etc., then I post on /images).
          > > In the parameters of the post request I would put a list of all the
          > > imaged ids (or entire uri), so that the server can handle the request
          > > atomically.
          >
          > I'd define a way to refer to a specific set of images. Say you want
          > to modify images 1, 4, and 13. You can PUT (and GET) to
          > "/images/1,4,13" with an entity fully describing three images. This
          > is just like your proposed idea, except you're referring to a limited
          > set of images instead of all of them. The server then atomically
          > replaces images 1, 4, and 13 with the entities you provided.
          >
          > One thing to note is that URIs like this require more complex linking.
          > The client needs to be able to construct those "1,4,13" URIs somehow,
          > but it must be told how to, at runtime, by the server; otherwise it
          > would violate HATEoAS. You need a way to say something like <a
          > href="/images/\d(,\d)*">, where the href there is actually a regex
          > describing how to build a URI out of a series of integers. Maybe
          > someone else can chime on how to do this well, because I have no idea.
          > I do know that you do *not* want to actually use regexes, though. :)
          >
          > > But then what should I get in return? What should /images return if I
          > > issued a GET? The whole repository? Naaa... too big.
          > > Isn't the POST supposed to return the same representation? Just after
          > > handling the posted request.
          >
          > This question is answered by the limited-subset solution above. When
          > you GET "/images/1,4,13", you get back exactly what you'd expect -
          > images 1, 4, and 13, just as they were when you PUT them in.
          >
          > > How much am I missing from the rest picture?
          >
          > Well, the PUT/POST thing, but everyone makes that mistake at first. :)
          > I think another thing people often miss is that the URI space doesn't
          > have to be rigid or static. That's exactly the issue here. What
          > you're trying to do is modify a subset of a collection, so just say it
          > in the URI! :)
          >
          > --
          > Gary
          > http://blog.extracheese.org
          >
          > ------------------------------------
          >
          > Yahoo! Groups Links
          >
          >
          >
          >

          --
          Sent from Google Mail for mobile | mobile.google.com

          Leonardo Boiko
          http://namakajiri.net
        • Gary Bernhardt
          ... The specs don t have any URI length restrictions, but current browsers do, so it depends on what your client is. BitBacker, the system I ve used this
          Message 4 of 19 , May 2, 2008
          • 0 Attachment
            On Fri, May 2, 2008 at 8:31 PM, Leonardo Boiko <leoboiko@...> wrote:
            > Side note: the "list URIs" above, as a not-much-known convention, use
            > "," as separator when order matters and ";" when it doesn't. Note also
            > that URIs have a maximum lenght.

            The specs don't have any URI length restrictions, but current browsers
            do, so it depends on what your client is. BitBacker, the system I've
            used this technique in, regularly generates URLs over 10,000
            characters long. This works fine, since both the server and client
            sides are prepared to deal with that. However, it can get very
            annoying when manually reading the log files. :)

            --
            Gary
            http://blog.extracheese.org
          • Sebastien Lambla
            Microsoft has proposed a bulk format so you can process all your requests in one go, using multipart messages as the payload, with each message being an http
            Message 5 of 19 , May 3, 2008
            • 0 Attachment
              Microsoft has proposed a bulk format so you can process all your requests in
              one go, using multipart messages as the payload, with each message being an
              http request. That way you can PUT on each of those resources in one go if
              you so wish.

              See:

              http://blogs.msdn.com/astoriateam/archive/2008/04/06/batching-data-service-r
              equests.aspx

              I'm not sure yet what I think about that proposal but it certainly does
              solve the problem of batching requests much better than http pipelining does
              it.

              And I personally find the whole POSt to http://server/resource/1,2,3,4 to be
              akward, with low visibility and discoverability, but that's a matter of
              taste :)

              SerialSeb
              -----Original Message-----
              From: rest-discuss@yahoogroups.com [mailto:rest-discuss@yahoogroups.com] On
              Behalf Of Stefano Masini
              Sent: 02 May 2008 21:56
              To: rest-discuss@yahoogroups.com
              Subject: [rest-discuss] Updating metadata of multiple resources

              Hi,

              I'm still trying to figure rest out. Let's see if I hit a sweet spot
              or if I'm just missing something (more likely, please forgive me).

              I have several images, each with its own uri that have metadata.
              If I want to update the metadata of a given image I send a POST
              request on its uri. Right?
              What if I want to atomically update some metadata (say, a tag) for a
              group of images, how do I approach that?
              Sending multiple POST requests, one for each uri, is not atomic, so I
              can't accept it.

              I thought about posting on the common parent of all the image uris
              (say, if I have /images/1, /images/2, etc., then I post on /images).
              In the parameters of the post request I would put a list of all the
              imaged ids (or entire uri), so that the server can handle the request
              atomically.

              But then what should I get in return? What should /images return if I
              issued a GET? The whole repository? Naaa... too big.
              Isn't the POST supposed to return the same representation? Just after
              handling the posted request.

              How much am I missing from the rest picture?

              Thanks
              Stefano

              ------------------------------------

              Yahoo! Groups Links
            • Gary Bernhardt
              ... By definition, it s either discoverable or it s not REST. If the representation linking to that resource tells the client how to construct the URL, it s
              Message 6 of 19 , May 3, 2008
              • 0 Attachment
                On Sat, May 3, 2008 at 12:04 PM, Sebastien Lambla <seb@...> wrote:
                > Microsoft has proposed a bulk format so you can process all your requests in
                > one go, using multipart messages as the payload, with each message being an
                > http request. That way you can PUT on each of those resources in one go if
                > you so wish.
                >
                > See:
                >
                > http://blogs.msdn.com/astoriateam/archive/2008/04/06/batching-data-service-r
                > equests.aspx
                >
                > I'm not sure yet what I think about that proposal but it certainly does
                > solve the problem of batching requests much better than http pipelining does
                > it.
                >
                > And I personally find the whole POSt to http://server/resource/1,2,3,4 to be
                > akward, with low visibility and discoverability, but that's a matter of
                > taste :)

                By definition, it's either discoverable or it's not REST. If the
                representation linking to that resource tells the client how to
                construct the URL, it's fully discoverable. If it fails to tell the
                client how, then the client has no way to construct the URL at all
                without violating HATEoAS.

                Now, if by "low visibility and discoverability" you mean "doesn't
                conform directly to the rigid 1990s view of the URL space", then sure.
                REST does require a bit of a mental leap. But I don't find "REST is
                hard, let's go shopping!" a very convincing argument. :)

                I've thought a lot about the idea of nesting multiple requests in a
                multipart payload. The more I've come to understand REST, the less
                useful that technique has seemed. If examples like the one above are
                motivating it, it's tremendous overkill. The URL space does not have
                to be rigid! Free your mind! :)

                --
                Gary
                http://blog.extracheese.org
              • Leonardo Boiko
                ... I don t feel that PUT is a natural way of assign folksonomy tags to pictures; POSTing tag URIs to an image resource or vice-versa feels more right to
                Message 7 of 19 , May 3, 2008
                • 0 Attachment
                  On Sat, May 3, 2008 at 1:04 PM, Sebastien Lambla <seb@...> wrote:
                  > That way you can PUT on each of those resources in one go if
                  > you so wish.

                  I don't feel that PUT is a natural way of assign "folksonomy" tags to
                  pictures; POSTing tag URIs to an image resource or vice-versa feels
                  more "right" to me. But it may be just me.

                  --
                  Leonardo Boiko
                  http://namakajiri.net
                • Sebastien Lambla
                  ... Metadata is probably a resource about a resource, so it could be said that it is in itself a resource that can be PUT to for updates, or POSTed to the
                  Message 8 of 19 , May 3, 2008
                  • 0 Attachment
                    > I don't feel that PUT is a natural way of assign "folksonomy" tags to
                    > pictures; POSTing tag URIs to an image resource or vice-versa feels
                    > more "right" to me. But it may be just me.

                    Metadata is probably a resource about a resource, so it could be said that it is in itself a resource that can be PUT to for updates, or POSTed to the image as a sub resource (rather than a child as I've seen it called earlier).

                    Same could be said about rdf documents

                    --
                    SerialSeb
                  • dev
                    Hi I don t get whats wrong with just using a POST on some meaningful URI that will update all the metadata you want. According to HTTP RFC : POST is for -
                    Message 9 of 19 , May 3, 2008
                    • 0 Attachment
                      Hi

                      I don't get whats wrong with just using a POST on some meaningful URI that will update all the metadata you want.

                      According to HTTP RFC :
                      POST is for
                            - Annotation of existing resources;

                      - Posting a message to a bulletin board, newsgroup, mailing list,
                      or similar group of articles;

                      - Providing a block of data, such as the result of submitting a
                      form, to a data-handling process;

                      - Extending a database through an append operation.

                      You seem to be doing 1,probably 3 and seems to be 4 too.

                      Regards
                      Devdatta


                      On 03/05/2008, Stefano Masini <stefano.masini@...> wrote:

                      Hi,

                      I'm still trying to figure rest out. Let's see if I hit a sweet spot
                      or if I'm just missing something (more likely, please forgive me).

                      I have several images, each with its own uri that have metadata.
                      If I want to update the metadata of a given image I send a POST
                      request on its uri. Right?
                      What if I want to atomically update some metadata (say, a tag) for a
                      group of images, how do I approach that?
                      Sending multiple POST requests, one for each uri, is not atomic, so I
                      can't accept it.

                      I thought about posting on the common parent of all the image uris
                      (say, if I have /images/1, /images/2, etc., then I post on /images).
                      In the parameters of the post request I would put a list of all the
                      imaged ids (or entire uri), so that the server can handle the request
                      atomically.

                      But then what should I get in return? What should /images return if I
                      issued a GET? The whole repository? Naaa... too big.
                      Isn't the POST supposed to return the same representation? Just after
                      handling the posted request.

                      How much am I missing from the rest picture?

                      Thanks
                      Stefano


                    • Sebastien Lambla
                      ... I fail to see how the server can instruct the client in any meaningful way that urls in the form of http://server/resource/1,2,3,4 represent a pool of
                      Message 10 of 19 , May 3, 2008
                      • 0 Attachment
                        > By definition, it's either discoverable or it's not REST. If the
                        > representation linking to that resource tells the client how to
                        > construct the URL, it's fully discoverable. If it fails to tell the
                        > client how, then the client has no way to construct the URL at all
                        > without violating HATEoAS.

                        I fail to see how the server can instruct the client in any meaningful way
                        that urls in the form of http://server/resource/1,2,3,4 represent a pool of
                        documents equal to /resource/1, /resource/2 etc. If you want to set that
                        convention up thats fine with me, but it's just that, a convention specific
                        to the way you define your service, and in that it is opaque with low
                        discoverability.

                        > REST does require a bit of a mental leap. But I don't find "REST is
                        > hard, let's go shopping!" a very convincing argument. :)

                        I don't believe I'm making this argument, quite the contrary. The solution
                        you seem to be promoting isn't more rest than batching requests in one go
                        through a multipart message. At least with that solution the semantics of
                        Urls are respected and still general purpose, as opposed to
                        convention-driven url construction.

                        > The URL space does not have to be rigid! Free your mind! :)

                        They have to be discoverable and not built by private conventions. I don't
                        believe in the value of transient composite Urls that have only meaning when
                        in the context of the documentation of your service, be it that it is 1,2,3
                        or 1;2;3.

                        If you start doing /1,2,3 supporting POST that translates in 3 POSTS to /1,
                        /2 and /3, do you support a GET on the same conventional URL? If not you
                        don't have a uniform interface anymore. If you do, you cannot support things
                        like ETags, caching etc, unless you do it as a collection.

                        Overall it seems to me it introduces more mess than it solves issues.

                        --
                        SerialSeb
                      • Gary Bernhardt
                        (Apologies for top posting; my response was getting huge and fragmented, so I summarized.) Let s get specific. If you re going to implement my method, you
                        Message 11 of 19 , May 3, 2008
                        • 0 Attachment
                          (Apologies for top posting; my response was getting huge and
                          fragmented, so I summarized.)

                          Let's get specific. If you're going to implement my method, you need
                          a new media type. Here's one possible implementation. Call it
                          "text/html-with-javascript-links". We add an extension to the <a> tag
                          so you can do things like this:

                          <a href="javascript:function(ids) { return '/resources/' + ids.join(','); }">

                          That little javascript function will return a string of the form
                          "/resources/1,2,3,4" when called with the list [1, 2, 3, 4]. This is
                          still perfectly RESTful, because the server is telling the client
                          exactly how to construct the URL. This is not a "convention" any more
                          than HTML forms are; it's just another media type. It's not "specific
                          to my service" because, once again, it's a (hypothetical) standard
                          media type. If you support it once, you support it for every service.
                          It's not "opaque" or "undiscoverable" because you have to have a
                          priori knowledge of it to even negotiate for a representation in the
                          first place.

                          It's also *much* simpler to implement than the proposed multi-part
                          request solution. Current clients support neither of the proposed
                          methods. I happen to be most familiar with Python's httplib, and I
                          can say without a doubt that multi-part requests would require serious
                          changes, since you can no longer make the assumption that HTTP
                          requests are of the form "send request, then block until the response
                          comes". Httplib makes exactly that assumption.

                          Contrast that with my proposed solution - all you're doing is
                          constructing the URL a little differently. There are no underlying
                          HTTP library changes. I can testify that this is very simple to do,
                          because I'm doing it right now, in a production system! It took maybe
                          an hour to add my first aggregate resource, and most of that time was
                          spent on the server-side business logic. It's been in production for
                          about six months and I've not given it a thought since; it's always
                          just worked. The media-type issues are separate, of course; I haven't
                          tackled them yet.

                          Aside from implementation complexity, the multi-part request solution
                          breaks HTTP in numerous subtle ways. It throws "one request, one
                          response" out the window. It also brings up thorny questions, like
                          "What happens if the 47th request in the series fails? How do you
                          tell the client? How does he recover?" The client now has a
                          fragmented concept of an "HTTP request" - sometimes a request is a
                          request; sometimes a request is actually a batch of requests. This
                          will trickle its way down into the client logic, not just the HTTP
                          library.

                          The application programmer will have to *explicitly* batch requests.
                          When he's batching request #13 out of 17, what is his current state?
                          What is the representation of that state? He sure as heck didn't get
                          it from the server! If your client states don't correspond one-to-one
                          with representations from the server, you're not doing
                          "representational state transfer"; you're just doing "state transfer"!
                          It ain't REST! :)

                          There's also the fundamental question of which method to use for your
                          multi-part request. Do you use POST? What if the batch of requests
                          is made up only of DELETEs? Is "delete these 57 resources" within the
                          definition of POST? Do you really have a uniform interface at that
                          point, or is your multi-part request really RPC in disguise? These
                          are the questions that made me back away from the "multi-part request"
                          technique, although it was initially tempting.

                          Sometimes widening your solution space a little is easier than digging
                          1,000 feet deeper into the hole you're already in. :)

                          --
                          Gary
                          http://blog.extracheese.org

                          On Sat, May 3, 2008 at 1:48 PM, Sebastien Lambla <seb@...> wrote:
                          > > By definition, it's either discoverable or it's not REST. If the
                          > > representation linking to that resource tells the client how to
                          > > construct the URL, it's fully discoverable. If it fails to tell the
                          > > client how, then the client has no way to construct the URL at all
                          > > without violating HATEoAS.
                          >
                          > I fail to see how the server can instruct the client in any meaningful way
                          > that urls in the form of http://server/resource/1,2,3,4 represent a pool of
                          > documents equal to /resource/1, /resource/2 etc. If you want to set that
                          > convention up thats fine with me, but it's just that, a convention specific
                          > to the way you define your service, and in that it is opaque with low
                          > discoverability.
                          >
                          >
                          > > REST does require a bit of a mental leap. But I don't find "REST is
                          > > hard, let's go shopping!" a very convincing argument. :)
                          >
                          > I don't believe I'm making this argument, quite the contrary. The solution
                          > you seem to be promoting isn't more rest than batching requests in one go
                          > through a multipart message. At least with that solution the semantics of
                          > Urls are respected and still general purpose, as opposed to
                          > convention-driven url construction.
                          >
                          >
                          > > The URL space does not have to be rigid! Free your mind! :)
                          >
                          > They have to be discoverable and not built by private conventions. I don't
                          > believe in the value of transient composite Urls that have only meaning when
                          > in the context of the documentation of your service, be it that it is 1,2,3
                          > or 1;2;3.
                          >
                          > If you start doing /1,2,3 supporting POST that translates in 3 POSTS to /1,
                          > /2 and /3, do you support a GET on the same conventional URL? If not you
                          > don't have a uniform interface anymore. If you do, you cannot support things
                          > like ETags, caching etc, unless you do it as a collection.
                          >
                          > Overall it seems to me it introduces more mess than it solves issues.
                        • Sebastien Lambla
                          ... This is where I think we differ: I believe infrastructure issues should be dealt at the infrastructure level, not pushed at the app level. By defining your
                          Message 12 of 19 , May 3, 2008
                          • 0 Attachment
                            > Let's get specific. If you're going to implement my method, you need
                            > a new media type. Here's one possible implementation. Call it
                            > "text/html-with-javascript-links".


                            This is where I think we differ: I believe infrastructure issues should be
                            dealt at the infrastructure level, not pushed at the app level. By defining
                            your javascript / html, you are rest, no doubt about it. But you do not
                            solve the problem of sending a whole bunch of requests that you would like
                            see processed in one block.

                            What is not provided in either solutions is to decide the behaviour on
                            failure: Updating on resources 1,2,3 would may fail for resource 2. What
                            happens then? Stop all of them, considering 1,2,3 were modified in one
                            transaction? Do you continue to 3 even though 2 failed? Do you stop at 2 and
                            leave 1 having been modified? And how do you communicate the result? By
                            wrapping several changes in one url, you can leverage the http status code
                            only in an atomic fashion, which may or may not be what your application
                            needs. The multipart approach lets you differentiate should you need it, and
                            only buy into the atomic change when and if you need it.

                            > [...] This is not a "convention" any more
                            > than HTML forms are; it's just another media type. It's not "specific
                            > to my service" because, once again, it's a (hypothetical) standard
                            > media type.

                            You may well be right in the sense that you rely on an informational
                            resource to tell you how to construct a url. That said, I fail to see how
                            this could be implemented easily in most toolkits: you rely on either having
                            prior knowledge or executing javascript. In that respect I don't see the
                            difference between your solution and the multipart one.

                            > If you support it once, you support it for every service.

                            I don't understand this assertion? Surely, any solution you come up with
                            will be generalizable to any resource, be it pipelining extended to
                            PUT/POST, multipart or your javascript-based URL building?

                            > It's not "opaque" or "undiscoverable" because you have to have a
                            > priori knowledge of it to even negotiate for a representation in the
                            > first place.

                            You need to understand the resource type, just like you need to understand
                            the batch multipart format, so in this respect I don't think you're adding
                            much value.

                            > It's also *much* simpler to implement than the proposed multi-part
                            > request solution. Current clients support neither of the proposed
                            > methods.

                            It may be a bit simpler to implement, but multipart mime is a known format
                            with plenty of supporting libraries supporting it. In itself it wouldn't
                            take much to support that bulk format from javascript, but certainly much
                            more than your solution.

                            > since you can no longer make the assumption that HTTP
                            > requests are of the form "send request, then block until the response
                            > comes". Httplib makes exactly that assumption.

                            I'm not sure we're talking about hte same proposal. All the requests are
                            sent in one go, and all the response are received in one go. On the wire you
                            have a simple request / response, so you can still do blocking stuff.

                            > It throws "one request, one
                            > response" out the window.

                            I don't understand how does it do that? You still get one response per
                            request, they just happened to be sent and received in bulk?

                            > "What happens if the 47th request in the series fails?

                            See my point earlier: this is the space of convention in both solutions.

                            > [...] sometimes a request is a
                            > request; sometimes a request is actually a batch of requests.

                            See above, n requests => n responses, this doesn't change this. What changes
                            is that you transmit all your requests as a payload.

                            > The application programmer will have to *explicitly* batch requests.
                            > When he's batching request #13 out of 17, what is his current state?

                            You assume that each set of changes is dependent on the new representation
                            you get after the previous operation. When you update the two picture's
                            metadata, you will receive an updated representation or a 204, be it that
                            you did a bulk update through url construction or through bulking the
                            requests. I really fail to see how your solution would be REST and the
                            multipart would only be ST, as the only difference is that you rely on
                            understanding a specific representation to build infrastructure bulk
                            changes, and multipart would rely on... The same thing! without the need for
                            the transient urls.

                            > There's also the fundamental question of which method to use for your
                            > multi-part request. Do you use POST? What if the batch of requests
                            > is made up only of DELETEs? Is "delete these 57 resources" within the
                            > definition of POST? Do you really have a uniform interface at that
                            > point, or is your multi-part request really RPC in disguise? These
                            > are the questions that made me back away from the "multi-part request"
                            > technique, although it was initially tempting.

                            That is certainly a shortcoming of the current solution and I agree with
                            you: is defining a _service_ called $batch supporting a post and always
                            returing a 200 the right thing to do? I'm not bothered too much about the
                            operations that are contained in the payload, as this url serves essentially
                            as a proxy to execute all the attached requests as if they were individual.

                            > Sometimes widening your solution space a little is easier than digging
                            > 1,000 feet deeper into the hole you're already in. :)

                            You assume that the batching requirements can be resolved simply by
                            combining resources of a same type and building transient uri through a
                            language sent down to the client, and that all those operations will always
                            be one operation and one type of operation. That's a lot of assumptions that
                            don't necessarily apply. This is a good solution for the small scope of the
                            issue that was presented though, but it doesn't resolve the more global
                            problem of batching a bunch of operations against n resources without
                            issuing n connections and waiting for the response n*m seconds.

                            In that respect the multipart approach solves the same kind of issues Http
                            pipelining should have solved for GET, as well as provide a way to group a
                            bunch of http operations in one unit of work.

                            That said the solution doesn't make me feel warm and fuzzy inside, but
                            neither does yours.

                            --
                            SerialSeb
                          • Subbu Allamaraju
                            While this approach looks better than pipelining, it is more like SOAP than HTTP. What is the difference between this, and say, packaging the same parts as
                            Message 13 of 19 , May 4, 2008
                            • 0 Attachment
                              While this approach looks better than pipelining, it is more like SOAP
                              than HTTP. What is the difference between this, and say, packaging the
                              same parts as some XML elements inside some envelope element?

                              Regards,
                              Subbu

                              On May 3, 2008, at 9:04 AM, Sebastien Lambla wrote:
                              > Microsoft has proposed a bulk format so you can process all your
                              > requests in
                              > one go, using multipart messages as the payload, with each message
                              > being an
                              > http request. That way you can PUT on each of those resources in one
                              > go if
                              > you so wish.
                              >
                              > See:
                              >
                              > http://blogs.msdn.com/astoriateam/archive/2008/04/06/batching-data-service-r
                              > equests.aspx
                              >
                              > I'm not sure yet what I think about that proposal but it certainly
                              > does
                              > solve the problem of batching requests much better than http
                              > pipelining does
                              > it.
                              >
                              > And I personally find the whole POSt to http://server/resource/
                              > 1,2,3,4 to be
                              > akward, with low visibility and discoverability, but that's a matter
                              > of
                              > taste :)
                              >
                              > SerialSeb
                              > -----Original Message-----
                              > From: rest-discuss@yahoogroups.com [mailto:rest-discuss@yahoogroups.com
                              > ] On
                              > Behalf Of Stefano Masini
                              > Sent: 02 May 2008 21:56
                              > To: rest-discuss@yahoogroups.com
                              > Subject: [rest-discuss] Updating metadata of multiple resources
                              >
                              > Hi,
                              >
                              > I'm still trying to figure rest out. Let's see if I hit a sweet spot
                              > or if I'm just missing something (more likely, please forgive me).
                              >
                              > I have several images, each with its own uri that have metadata.
                              > If I want to update the metadata of a given image I send a POST
                              > request on its uri. Right?
                              > What if I want to atomically update some metadata (say, a tag) for a
                              > group of images, how do I approach that?
                              > Sending multiple POST requests, one for each uri, is not atomic, so I
                              > can't accept it.
                              >
                              > I thought about posting on the common parent of all the image uris
                              > (say, if I have /images/1, /images/2, etc., then I post on /images).
                              > In the parameters of the post request I would put a list of all the
                              > imaged ids (or entire uri), so that the server can handle the request
                              > atomically.
                              >
                              > But then what should I get in return? What should /images return if I
                              > issued a GET? The whole repository? Naaa... too big.
                              > Isn't the POST supposed to return the same representation? Just after
                              > handling the posted request.
                              >
                              > How much am I missing from the rest picture?
                              >
                              > Thanks
                              > Stefano
                              >
                              > ------------------------------------
                              >
                              > Yahoo! Groups Links
                              >
                              >
                              >
                              >
                            • Sebastien Lambla
                              ... Several benefits: - Multipart messages are a standard way to represent compound payloads - You don t enforce the use of xml If you were to compound those
                              Message 14 of 19 , May 4, 2008
                              • 0 Attachment
                                > While this approach looks better than pipelining, it is more like SOAP
                                > than HTTP. What is the difference between this, and say, packaging the
                                > same parts as some XML elements inside some envelope element?

                                Several benefits:
                                - Multipart messages are a standard way to represent compound payloads
                                - You don't enforce the use of xml

                                If you were to compound those requests in a specific xml document, you need
                                to define a custom mime type rather than reuse an existing one, force the
                                understanding of the compound payload on the consumer, and you will end up
                                having to represent the semantics of multiple requests by serializing your
                                http message in one way or another, at which point it makes more sense from
                                my perspective to just package the message in its natural format
                                (application/http is defined in rfc2616).

                                Where you are right is that the $batch is the definition of a service, and
                                some would argue that services have no place in a rest architecture, because
                                all visible area should be resources. I don't think it's inherently bad to
                                have some resources be services that have no semantic value, I am just
                                concerned that you lose a lot of discoverability and you create orthogonal
                                concerns when dealing with an API.

                                To be clear on my position on this proposal: I really like the idea of
                                multipart of application/http and get the result in the same format. I am
                                however not certain I like the semantics of the $batch url and the semantics
                                associated with that endpoint, how discoverable it is and how knowledge of
                                the transfer and format is sent to the client, so it's mixed reaction from
                                me. But I don't like other proposals for the reasons i've invoked in my
                                previous emails and the inherent flexibility of the multipart approach is
                                very seductive.

                                --
                                SerialSeb
                              • Stefano Masini
                                First of all thank you all for the great responses this thread got. They were very helpful to a starter like me. So thank you. But now let me get this
                                Message 15 of 19 , May 4, 2008
                                • 0 Attachment
                                  First of all thank you all for the great responses this thread got.
                                  They were very helpful to a starter like me. So thank you.

                                  But now let me get this straight. I hope you'll forgive my naive questions.

                                  On Sun, May 4, 2008 at 2:39 AM, Sebastien Lambla <seb@...> wrote:
                                  > > than HTML forms are; it's just another media type. It's not "specific
                                  > > to my service" because, once again, it's a (hypothetical) standard
                                  > > media type.
                                  >
                                  > You may well be right in the sense that you rely on an informational
                                  > resource to tell you how to construct a url. That said, I fail to see how
                                  > [...]

                                  I'll hook up here, and try to go back to my original example.

                                  So I think I understand (and like) the principle of discoverability,
                                  in the sense that the client should not be held responsible to come up
                                  with urls. Instead it should somehow be told about them by the server.
                                  The client remains responsible for is the _why_ a given url is needed,
                                  but the _how_ is told by the server. Good.

                                  So, back to my image repository example: let's assume that the
                                  preferred representation is Json and that the client wants to make use
                                  of a url identifying a given set of images. Gary proposed an approach
                                  making use of a "text/html-with-javascript-links" media type, with the
                                  representation being some html containing a javascript code snippet.
                                  My first question: can I safely translate that proposal to my example
                                  by rewriting that code snippet as the following Json data structure?
                                  {
                                  imageSetUrlBuilder: "function(ids) { return '/resources/' + ids.join(','); }"
                                  }

                                  The client application, of course, should know how to deal with such
                                  information, just like it should know about the
                                  text/html-with-javascript-links media type, or anything else. I'd say,
                                  it actually _expects_ it, rather than just knowing how to deal with
                                  it.

                                  Now my second question: which part of my url space is better suited
                                  for containing this piece of Json?
                                  If I think about it, it seems like some sort of application global
                                  "information". It's a piece of functionality (in the form of a
                                  javascript code snippet) that the client app needs to fetch right at
                                  the beginning in order to be fully functional. So, I'd say, the root
                                  url (/) is the best place, together with all the other stuff I might
                                  like to include in the root url, i.e. the one representing my
                                  application in general, including other urls. For example:
                                  {
                                  imageSetUrlBuilder: "function(ids) { return '/resources/' + ids.join(','); }",
                                  allImagesUrl: "/images",
                                  title: "My fancy application"
                                  }

                                  What do you think? Am I beginning to see more RESTy? :)

                                  Thanks
                                  Stefano
                                • Subbu Allamaraju
                                  ... I don t disagree. But today s infrastructure does not deal with POST, PUT etc packaged inside multipart messages. For instance, a cache can t mark an entry
                                  Message 16 of 19 , May 4, 2008
                                  • 0 Attachment
                                    On May 4, 2008, at 8:39 AM, Sebastien Lambla wrote:
                                    >> While this approach looks better than pipelining, it is more like
                                    >> SOAP
                                    >> than HTTP. What is the difference between this, and say, packaging
                                    >> the
                                    >> same parts as some XML elements inside some envelope element?
                                    >
                                    > Several benefits:
                                    > - Multipart messages are a standard way to represent compound payloads
                                    > - You don't enforce the use of xml

                                    I don't disagree. But today's infrastructure does not deal with POST,
                                    PUT etc packaged inside multipart messages. For instance, a cache
                                    can't mark an entry in the cache stale when a PUT is submitted via a
                                    multipart message. Yes, there are theoretical benefits, but it is
                                    difficult to realize those benefits in reality.

                                    The basic motivation for proposals like this is to let a client submit
                                    a bunch of requests and get them executed in at once. The supposed
                                    benefit for the client is that it does not have to make "n" number of
                                    requests. However, IMO, once we get into the details, the picture
                                    won't be as simple as this. For instance, what is the cost for the
                                    client to recover from partial failures? What is cost for the client
                                    and server to use two different code paths - one for regular requests
                                    and another for batch requests? Does the client sufficiently
                                    understand the difference between non-idempotent requests and
                                    idempotent requests bundled in the same batch? For the simple cases,
                                    yes, the answers may be simple, but IMO, the complexity of batch may
                                    outweigh the benefits.

                                    Moreover, in most cases, even if pipelining could not be replied upon
                                    for whatever reasons, by modeling resources cleverly, it may be
                                    possible to avoid the need for batch.

                                    Sincerely,
                                    Subbu
                                  • Bill de hOra
                                    ... For AtomPub servers, I implement this for batch tagging: POST /categorise host: example.org ...
                                    Message 17 of 19 , May 6, 2008
                                    • 0 Attachment
                                      Stefano Masini wrote:
                                      >
                                      >
                                      > Hi,
                                      >
                                      > I'm still trying to figure rest out. Let's see if I hit a sweet spot
                                      > or if I'm just missing something (more likely, please forgive me).
                                      >
                                      > I have several images, each with its own uri that have metadata.
                                      > If I want to update the metadata of a given image I send a POST
                                      > request on its uri. Right?
                                      > What if I want to atomically update some metadata (say, a tag) for a
                                      > group of images, how do I approach that?
                                      > Sending multiple POST requests, one for each uri, is not atomic, so I
                                      > can't accept it.

                                      For AtomPub servers, I implement this for batch tagging:

                                      POST /categorise
                                      host: example.org

                                      <categories ...>
                                      <category term="foo" >
                                      <atom:id>...</atom:id>
                                      <atom:id>...</atom:id>
                                      <atom:id>...</atom:id>
                                      </category>
                                      <category term="foo" >
                                      <atom:id>...</atom:id>
                                      <atom:id>...</atom:id>
                                      <atom:id>...</atom:id>
                                      </category>
                                      </categories>

                                      and have a symmetric uncategorise service for removing tags. I'm still
                                      working on batch title edits, which is the other common usecase I run into.

                                      Bill
                                    • Steve Loughran
                                      ... 1. There is some bulk operation in WebDAV, which then requires the notion of partial failure to come back in the response; a simple single error code is no
                                      Message 18 of 19 , May 7, 2008
                                      • 0 Attachment
                                        On Mon, May 5, 2008 at 12:44 AM, Subbu Allamaraju <subbu@...> wrote:
                                        >
                                        > On May 4, 2008, at 8:39 AM, Sebastien Lambla wrote:
                                        > >> While this approach looks better than pipelining, it is more like
                                        > >> SOAP
                                        > >> than HTTP. What is the difference between this, and say, packaging
                                        > >> the
                                        > >> same parts as some XML elements inside some envelope element?
                                        > >
                                        > > Several benefits:
                                        > > - Multipart messages are a standard way to represent compound payloads
                                        > > - You don't enforce the use of xml
                                        >
                                        > I don't disagree. But today's infrastructure does not deal with POST,
                                        > PUT etc packaged inside multipart messages. For instance, a cache
                                        > can't mark an entry in the cache stale when a PUT is submitted via a
                                        > multipart message. Yes, there are theoretical benefits, but it is
                                        > difficult to realize those benefits in reality.
                                        >
                                        > The basic motivation for proposals like this is to let a client submit
                                        > a bunch of requests and get them executed in at once. The supposed
                                        > benefit for the client is that it does not have to make "n" number of
                                        > requests. However, IMO, once we get into the details, the picture
                                        > won't be as simple as this. For instance, what is the cost for the
                                        > client to recover from partial failures? What is cost for the client
                                        > and server to use two different code paths - one for regular requests
                                        > and another for batch requests? Does the client sufficiently
                                        > understand the difference between non-idempotent requests and
                                        > idempotent requests bundled in the same batch? For the simple cases,
                                        > yes, the answers may be simple, but IMO, the complexity of batch may
                                        > outweigh the benefits.

                                        1. There is some bulk operation in WebDAV, which then requires the
                                        notion of partial failure to come back in the response; a simple
                                        single error code is no longer enough

                                        2. WS-ResourceFramework, while near the pinnacle of WS-*, tried to
                                        retrofit bulk operations to its resource-model-atop-SOAP world view,
                                        and again, got into a mess about atomicity of updates as well as
                                        failures. What I dont like about it (apart from it being the pinnacle
                                        of WS-*) is that there was no per-resource metadata about the
                                        atomicity, and no guarantee that all servers worked the same.

                                        the nice thing about doing bulk operations as a sequence of requests
                                        is that you can be sure that
                                        -the bulk operation is not atomic
                                        -if one operation fails, everything you havent yet pushed out wont
                                        have gone through
                                        -state changes are observable to others during the sequence of operations.
                                        -the single update code path==the bulk update code path
                                        It may not be the exact behaviour you want, but at least you can be
                                        sure that it will be consistent, that no server updates will suddenly
                                        change their behaviour without you noticing. Which is good, as it is
                                        surprisingly hard to write tests to determine exactly what the bulk
                                        update semantics of a remote system are.

                                        -steve
                                      • Subbu Allamaraju
                                        I agree. ... Exactly, that is why I compared this to SOAP. ... Well said. Subbu
                                        Message 19 of 19 , May 7, 2008
                                        • 0 Attachment
                                          I agree.

                                          > 1. There is some bulk operation in WebDAV, which then requires the
                                          > notion of partial failure to come back in the response; a simple
                                          > single error code is no longer enough

                                          Exactly, that is why I compared this to SOAP.

                                          > 2. WS-ResourceFramework, while near the pinnacle of WS-*, tried to
                                          > retrofit bulk operations to its resource-model-atop-SOAP world view,
                                          > and again, got into a mess about atomicity of updates as well as
                                          > failures. What I dont like about it (apart from it being the pinnacle
                                          > of WS-*) is that there was no per-resource metadata about the
                                          > atomicity, and no guarantee that all servers worked the same.
                                          >

                                          :)

                                          > the nice thing about doing bulk operations as a sequence of requests
                                          > is that you can be sure that
                                          > -the bulk operation is not atomic
                                          > -if one operation fails, everything you havent yet pushed out wont
                                          > have gone through
                                          > -state changes are observable to others during the sequence of
                                          > operations.
                                          > -the single update code path==the bulk update code path
                                          > It may not be the exact behaviour you want, but at least you can be
                                          > sure that it will be consistent, that no server updates will suddenly
                                          > change their behaviour without you noticing. Which is good, as it is
                                          > surprisingly hard to write tests to determine exactly what the bulk
                                          > update semantics of a remote system are.

                                          Well said.

                                          Subbu
                                        Your message has been successfully submitted and would be delivered to recipients shortly.