Loading ...
Sorry, an error occurred while loading the content.

Re: [rest-discuss] modelling transactions

Expand Messages
  • Paul Winkler
    ... Thanks for the link, but AFAICT it s not really useful if neither the client or the server is Java ;-) Nearly everything here is either Ruby or Python
    Message 1 of 17 , Feb 2, 2007
    • 0 Attachment
      On Fri, Feb 02, 2007 at 09:59:31AM +0100, Jan Algermissen wrote:
      > You might also want to check out JSR 170[1]

      Thanks for the link, but AFAICT it's not really useful
      if neither the client or the server is Java ;-)
      Nearly everything here is either Ruby or Python nowadays.

      --

      Paul Winkler
      http://www.slinkp.com
    • Harley Pebley
      ... Be careful in discounting it just because of that. I ve found reading through some JSRs to be very helpful even though I don t do Java. They can sometimes
      Message 2 of 17 , Feb 2, 2007
      • 0 Attachment
        > > You might also want to check out JSR 170[1]
        >
        >Thanks for the link, but AFAICT it's not really useful
        >if neither the client or the server is Java ;-)

        Be careful in discounting it just because of that. I've
        found reading through some JSRs to be very helpful even
        though I don't do Java. They can sometimes give you good
        design ideas. (I haven't looked at this particular one
        to know if it would in this situation or not...)

        Cheers,
        Harley Pebley
        www.skylark-software.com
      • Jan Algermissen
        ... Hmm, IMHO it is ok, since client and server share the semantics provided in the APP extension and if the extension defines a new HTTP header then they
        Message 3 of 17 , Feb 2, 2007
        • 0 Attachment

          On 02.02.2007, at 18:01, Paul Winkler wrote:


            POST /documents

            Transaction-URI: http://foo.org/transactions/3


          Okay. I hadn't looked into the idea of custom headers.  Somehow I

          assumed that wasn't kosher,


          Hmm, IMHO it is ok, since client and server share the semantics provided in the APP extension and if the extension defines a new HTTP header then they should both be aware of it. Others mustIgnore the header.

          Besides that, you could also include the Transaction URI in the Atom envelope (the extension could define an additional control element element)...well, except for DELETE requests.


          since it means the client is assuming

          non-standard capabilities on the server.


          Yes, it does. Bit it does so only because the server declared that it (well, the collection) supports a specific feature. It is this feature that is the *reason* for the client to pick that collection and not another one. Think of it as late binding of the client component to the server component based on capabilities.

          With SOAP/WSDL components late-bind based on the API, with HTTP/Atom they late-bind based on declared capabilities (or declared type if you want).

          Gues what is more flexible :-)
          (I'd also argue that the former involves the latter anyway and thus is just more work, unnecessary overhead, obsolete complexity....eventually wasted time and money). 


          I am still lacking in

          rest-fu.


          What is 'rest-fu' ?

          Jan



        • Benjamin Carlyle
          ... There are two basic approaches here: 1. Model a transaction over multiple small edits 2. Make a resource available that allows the user to operate on the
          Message 4 of 17 , Feb 3, 2007
          • 0 Attachment
            On Fri, 2007-02-02 at 00:16 -0500, Paul Winkler wrote:
            > I've never built a REST app but am thinking of going that direction
            > with the next major version of a content repository API used by my
            > employer. There's a few things I'm not sure how to model, the main
            > one being transactions. I'm hoping for some guidance on my path to
            > grokking REST.
            ...
            > But what about this use case? A third party comes along and licenses
            > 10,000 documents to us. There are lots of cross-links among those
            > documents, so a requirement in the contract is that we must load the
            > entire set successfully, or none at all.

            There are two basic approaches here:
            1. Model a transaction over multiple small edits
            2. Make a resource available that allows the user to operate on the
            entire state they want to modify as one request

            The former has various implementations, but is painful to implement.
            Part of that pain is because it butts up against the "stateless between
            requests" constraint of REST. You transmit a bit of what you want to do,
            then another bit in another request, and then finally request that all
            of the bits you requested actually happen. This may be necessary if the
            client has to consult current state during the transaction, however
            thinking of a REST service in the same way you think about a database is
            usually a mistake.

            My suggestion is to use a resource that allows them to do everything
            they want as one operation. Without a more detailed explaination of what
            the customer might be doing it is hard to be specific, but let's look at
            an example where they want to publish a thousand atom articles:

            >>>
            POST http://example.com/newentries
            <atom>
            <entry>
            entry 1
            </entry>
            <entry>
            entry 2
            </entry>
            ...
            <entry>
            entry 1000
            </entry>
            </atom>
            <<<
            200 OK

            You can use a POE technique to ensure that this operation can be safely
            retried if the client recieves no response.

            What about updating existing entries? Perhaps something like this would
            work:

            PUT http://example.com/select?entries=http://example.com/0001
            +http://example.com/0002...+http://example.com/1000
            <atom>
            <entry xml:base="http://example.com/0001">
            entry 1
            </entry>
            <entry xml:base="http://example.com/0002">
            entry 2
            </entry>
            ...
            <entry xml:base="http://example.com/1000">
            entry 1000
            </entry>
            </atom>

            Whether attributes to identify each entry being updated are needed or
            whether the order can be determined by the query is probably up for
            debate, as is whether or not xml:base has strong enough semantics to
            indicate which entry is which. This approach ultimately boils down to
            finding a url that represents all of the data you want to update,
            preferrably by filling out a server-provided template or form to locate
            the data.

            In my opinion, online services of all kinds (From REST to WS-*) will
            tend to perform, scale and behave better when a single request is a
            single request. Once the request is received, it is the responsbility of
            the service that recieves the request to ensure it is treated as an
            atomic update.

            Benjamin
          • Paul Winkler
            ... (snip) That s fine and dandy, but I m more concerned about the example where they want to publish ten thousand atom media entities. (snip) ... What is POE?
            Message 5 of 17 , Feb 5, 2007
            • 0 Attachment
              On Sun, Feb 04, 2007 at 10:55:43AM +1000, Benjamin Carlyle wrote:
              > My suggestion is to use a resource that allows them to do everything
              > they want as one operation. Without a more detailed explaination of what
              > the customer might be doing it is hard to be specific, but let's look at
              > an example where they want to publish a thousand atom articles:
              (snip)

              That's fine and dandy, but I'm more concerned about the example where
              they want to publish ten thousand atom media entities.

              (snip)
              > You can use a POE technique to ensure that this operation can be safely
              > retried if the client recieves no response.

              What is POE?

              > In my opinion, online services of all kinds (From REST to WS-*) will
              > tend to perform, scale and behave better when a single request is a
              > single request. Once the request is received, it is the responsbility of
              > the service that recieves the request to ensure it is treated as an
              > atomic update.

              Noted. Many of our requirements fit perfectly into the single-request
              model. It's just the bulk loading stuff that's a pain.
              Maybe I should provide as separate interface for that.

              --

              Paul Winkler
              http://www.slinkp.com
            • Nic James Ferrier
              ... Is it though? Surely you could publish a format requirement for a POSTed entity that allowed bulk uploading in one shot. -- Nic Ferrier
              Message 6 of 17 , Feb 5, 2007
              • 0 Attachment
                Paul Winkler <pw_lists@...> writes:

                > Noted. Many of our requirements fit perfectly into the single-request
                > model. It's just the bulk loading stuff that's a pain.
                > Maybe I should provide as separate interface for that.

                Is it though? Surely you could publish a format requirement for a
                POSTed entity that allowed bulk uploading in one shot.

                --
                Nic Ferrier
                http://www.tapsellferrier.co.uk for all your tapsell ferrier needs
              • Paul Winkler
                ... Sure, but that s impractical when I have gigabytes of stuff to load in one transaction , as mentioned previously in the thread. -- Paul Winkler
                Message 7 of 17 , Feb 5, 2007
                • 0 Attachment
                  On Mon, Feb 05, 2007 at 06:54:20PM +0000, Nic James Ferrier wrote:
                  > Paul Winkler <pw_lists@...> writes:
                  >
                  > > Noted. Many of our requirements fit perfectly into the single-request
                  > > model. It's just the bulk loading stuff that's a pain.
                  > > Maybe I should provide as separate interface for that.
                  >
                  > Is it though? Surely you could publish a format requirement for a
                  > POSTed entity that allowed bulk uploading in one shot.

                  Sure, but that's impractical when I have gigabytes of stuff to load in
                  one "transaction", as mentioned previously in the thread.

                  --

                  Paul Winkler
                  http://www.slinkp.com
                • Nic James Ferrier
                  ... Ok. But if you can t load it in a single request then.... -- Nic Ferrier http://www.tapsellferrier.co.uk for all your tapsell ferrier needs
                  Message 8 of 17 , Feb 5, 2007
                  • 0 Attachment
                    Paul Winkler <pw_lists@...> writes:

                    > On Mon, Feb 05, 2007 at 06:54:20PM +0000, Nic James Ferrier wrote:
                    >> Paul Winkler <pw_lists@...> writes:
                    >>
                    >> > Noted. Many of our requirements fit perfectly into the single-request
                    >> > model. It's just the bulk loading stuff that's a pain.
                    >> > Maybe I should provide as separate interface for that.
                    >>
                    >> Is it though? Surely you could publish a format requirement for a
                    >> POSTed entity that allowed bulk uploading in one shot.
                    >
                    > Sure, but that's impractical when I have gigabytes of stuff to load in
                    > one "transaction", as mentioned previously in the thread.

                    Ok. But if you can't load it in a single request then....


                    --
                    Nic Ferrier
                    http://www.tapsellferrier.co.uk for all your tapsell ferrier needs
                  • Benjamin Carlyle
                    ... Sorry: Post Once Exactly. Techniques for this are currently being discussed in the POST at most once thread. ... That would seem to be a problem with
                    Message 9 of 17 , Feb 6, 2007
                    • 0 Attachment
                      On Mon, 2007-02-05 at 10:25 -0500, Paul Winkler wrote:
                      > On Sun, Feb 04, 2007 at 10:55:43AM +1000, Benjamin Carlyle wrote:
                      > > You can use a POE technique to ensure that this operation can be
                      > safely
                      > > retried if the client recieves no response.
                      > What is POE?

                      Sorry: Post Once Exactly. Techniques for this are currently being
                      discussed in the POST at most once thread.

                      > On Mon, 2007-02-05 at 14:05 -0500, Paul Winkler wrote:
                      > On Mon, Feb 05, 2007 at 06:54:20PM +0000, Nic James Ferrier wrote:
                      > > Paul Winkler <pw_lists@...> writes:
                      > > > Noted. Many of our requirements fit perfectly into the
                      > single-request
                      > > > model. It's just the bulk loading stuff that's a pain.
                      > > > Maybe I should provide as separate interface for that.
                      > > Is it though? Surely you could publish a format requirement for a
                      > > POSTed entity that allowed bulk uploading in one shot.
                      > Sure, but that's impractical when I have gigabytes of stuff to load in
                      > one "transaction", as mentioned previously in the thread.

                      That would seem to be a problem with implementation technology, however
                      I conceed that such limitations need to be worked around :)

                      So let's look at this in stages:
                      1. Transfer state of new publications to server
                      2. Server acts on the publication as an atomic unit
                      3. Client is notified of transaction completion

                      The simplest implementation of POST -> 200 OK is not available because
                      the implementation technology would cause us to eat up all available
                      memory just transferring the request. So we use

                      1a. Use any available file transfer mechanism from client to server
                      1b. Request the server use the transferred file as input
                      2. Server acts on publication as an atomic unit
                      3. Server replies with 200 OK, or sends a request back to the client
                      saying effectively 200 OK.

                      This leaves us with a number of problems still to solve. For example,
                      how do we phrase the "load this data" request. It is most analagous to
                      the WebDav COPY request. ie, copy the file resource we just gave you to
                      the set of published nodes. You'll find varying opinions on the merit of
                      COPY on this list. IIRC Fielding has noted in the past that HTTP is
                      designed to operate on a single url, and that WebDav COPY and related
                      operations have both theoretical and practical problems attached to
                      them.

                      The other alternative is to head in the direction you are already
                      heading, which is unlikely to be worse than the file transfer + COPY
                      approach. Perform the transaction piecemeal with POSTs to a transaction
                      url or other requests that refer to the transaction url in a header,
                      then POST a commit marker into the transaction resource.

                      If the client really needs to make the update atomically and your
                      implementation technology really doesn't permit the update to be
                      performed as a single request, you have to head in this sort of
                      direction. Just plan your error recovery carefully. There are a lot more
                      ways that multiple requests can go wrong.

                      Benjamin.
                    • Paul Winkler
                      ... Indeed! Thanks very much for the tips. -- Paul Winkler http://www.slinkp.com
                      Message 10 of 17 , Feb 6, 2007
                      • 0 Attachment
                        On Wed, Feb 07, 2007 at 07:24:18AM +1000, Benjamin Carlyle wrote:
                        > .... Just plan your error recovery carefully. There are a lot more
                        > ways that multiple requests can go wrong.

                        Indeed! Thanks very much for the tips.

                        --

                        Paul Winkler
                        http://www.slinkp.com
                      Your message has been successfully submitted and would be delivered to recipients shortly.