Re: [rest-discuss] Search engine REST interface
- A small point, but in the introduction you link to RFC 2068 which is
obsoleted by RFC 2616.
Duncan Wilcox wrote:
> this is the documentation for a REST interface to a search engine I have
> working on:
> I'd appreciate any feedback you might have.
> One glaring problem is the use of a "magic" cookie instead of a much more
> RESTful use of ETags, in the search index configuration. Unfortunately the
> REST interface is running as a CGI, and the CGI environment doesn't give me
> control at the If-Match stage. I'm planning on fixing this either by going
> the Apache module way, or by rolling a mini web server.
> Other than general comments on RESTfullness, would you produce DTDs or
> for the XML documents? (yeah, XML is not a requirement of REST, it's just a
> useful convention) Would you output more useful error documents instead of
> just e.g. "406 'query' parameter can't be null"?
> *Yahoo! Groups Sponsor*
> To unsubscribe from this group, send an email to:
> Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service
- Below is a conversation I've had about how to do REST-ful Web
applications. I'd appreciate any comments that you think might
save me from ending up in the non-REST-ful gutter.
On Mon, Mar 31, 2003 at 11:35:36AM -0700, VanL wrote:
> Dave Kuhlman wrote:
> >If you have more questions, please let me know.
> Since you invite questions, I do have a few. I just read all of the
> above, as well as a considerable portion of the REST wiki. I picked up
> some valuable insights, but I am not sure how some of my application
> could be implemented in a REST-ful way. I would appreciate your insight
> about REST and how it solves (or doesn't solve) the problem below.
> For example: a billing application that is implemented as a "wizard" --
> a multi-page form, where each step in the process depends on the
> successful performance of previous steps. It also has the following
> - Because the completion of the wizard has side-effects (e.g., the
> billing of a credit card), that final action must not take place unless
> all previous portions of the wizard have been completed in the same
> session. As a corollary, the final action must not take place unless
> all previous steps have been completed.
> - The application must be able to tell what sections of the wizard have
> been completed successfully, and save that information somehow for the
> completion of the wizard. Even if someone bookmarks a page in the
> middle of the form and sends that URL to another person, the application
> must sense that a new session has started and transition back to the
> start stage.
> You probably know the type of application that I am referring to...
> almost every online shop has something like it. However, I cannot even
> think of a way to even define the application in a stateless (or at
> least idempotent) fashion.
> Do you have a solution to this problem?
Interesting problem. And, a very realistic one, too. Here are
some attempts at answers, but keep in mind that I am not a REST
First, I believe the claims about REST being state-less are very
misleading. After all, the "S" in "REST" stands for "state". The
server is state-less, but REST applications can have state. It's
just that maintaining state is the responsibility of the client,
not the server. Another way to say this is to say that the server
maintains no session information.
In the FSM/REST model that I use, state/session information is
returned from the server to the client in an XML document. In the
subsequent request, the client returns this XML document to the
server. So, state information is passed back and forth between
server and client.
I think you can see that the billing information could be passed
back and forth between the server and client (perhaps in an XML
document) until it is complete, error free, contains a valid credit
card number, etc.
Second, if as a requirement you are determined to keep the part of
the problem about a multi-part form and a multi-step process, then
something like the above, where state/session information is passed
back and forth between the client and server or even the FSM/REST
model I describe might be a solution.
But, third, let's leave the FSM/REST stuff aside, since that's a
little bit too much my own favorite solution. So, having said all
that, I believe that a serious REST developer might try to
re-design your application. Again, I'm not a REST expert, but let
me try. In your billing application, we'd define a resource,
perhaps called "Current billing request". This resource might be
stored as a set of records in an relational database on the server.
Once this resource has been created for a client, a URI for it
would be returned to the client. The client could then request,
edit, and POST to this resource as many times as desired. The
client does this by using the URI to request this resource and then
POST-ing changes against it. At any time in this process, the
client can attempt to submit or attempt to complete the billing
request. It is the responsibility of the server to only perform
this action if the billing information is complete, free from
errors, etc. Once it has been submitted successfully, the server
removes the "Current billing request" resource, creates a
"Submitted billing" resource, and returns a URI to the client for
that new resource.
I think what we have done in this re-design, is to take what we
might have called state, and turned it into a (temporary?)
A few additional comments -- It might be the case that there is a
requirement that the implementation follow an existing business
process (BP) closely. The type of redesign work I've suggested
might, therefore, be objectionable. I can especially imagine this
requirement if the BP is very complex, in which case a business
analyst might want, for example, to be able to check the
implementation for correctness by comparing it to the (back-end)
BP. Well, my work with FSMs and REST is an attempt to address that
issue. FSM/REST is an attempt to enable developers to expose
complex processes as REST-ful Web applications, just as long as the
BP can be modeled as an FSM, which, I suppose, might require some
re-design work itself (sigh).
And, going back and trying to get more into the spirit of your
example application with its multi-part process and multi-part
form: perhaps each part of the multi-part form could be a separate
resource which the client could retrieve and post updates to. At
the bottom of each page returned to the client is the URI of the
next part/page of the form to be edited and POST-ed. And, there
would also be a URI that the client could request in order to
submit the billing. Again, it is up to the server app to check
these resources for errors and completeness before submitting the
request to the credit card company.
Did I get anywhere close to a solution to your problem?
Thanks for the provocative example.
I'd like to post your example problem/application and my reply to
the REST email list. (http://groups.yahoo.com/group/rest-discuss/)
Would that be OK? Maybe someone there will have better ideas than
mine. It's certainly a problem that we'd need a good solution too
if REST is going to be taken seriously.
On Mon, Mar 31, 2003 at 09:26:14PM -0700, VanL wrote:
> Dave Kuhlman wrote:
> >Did I get anywhere close to a solution to your problem?
> I'm not sure, I have to think about it. Right off had, there are some
> issues that come to mind:
> 1. Never trust the client. I know that you can cryptographically sign
> the encoded state, and check signatures, but I worry that embedding the
> state in the request opens up Pandora's box a little too much.
One way to respond to this is to ask: Does this kind of design
enable the the server to be careful enough? Does it enable the
server to perform checks that make the application safe? I think
it does. But, that does place burden on the developer of the
server application, and I agree that you're justified in being
concerned about that.
> 2. The additional encoding-decoding of state is additional overhead that
> would not be required, should the state be kept server-side.
The alternative, maintaining session information on the server
requires cycles and storage, too.
And, unless your session requirements are simple, session
management takes human (development) cycles as well. A company I
used to work for developed a tool where we had to worry about when
sessions expired, about enabling customers to control the
expiration of sessions, about whether sessions could be maliciously
captured from another machine, etc.
> 3. In the design you mention, the URI does not completely embody the
> state of the app - it is a combination of URI+embedded/encoded state.
> I'm not sure that this is a problem, but if (my admittedly poor)
> understanding of the REST rules is correct, it is a violation.
Let me try to clarify what this design does, then we can decide
whether it is consistent with or violates REST:
1. The client has a URI that identifies a resource.
2. The client makes requests to the URI and with the request the
client submits input items (possibly some state information).
In the design I've suggested, the input is in an XML document
that is POST-ed with the request.
3. The server may be "building" resources as the client makes
requests. For example, the server may be building a client
record or a billing record.
So, let's ask: In what way does this violate REST?
Posting data to a URI is consistent with REST.
Updating a back-end resource from posted data is consistent with
REST. Although, I'll admit that the idempotent requirement
And, returning content as a result of a request is consistent with
But, I'll admit that the whole idea of state and doing FSM in a
REST-ful style is questionable. Perhaps it pushes the REST
> I brought up this billing app, but the problem is much more general -- I
> think that any app that undertakes an action that has side-effects is
> going to encounter this sort of issue.
Looks like the essential requirements for the problem are:
1. The application is a multi-step process. Filling out a
multi-part form (as in your billing application) is a specific
problem, but the problem is more general.
2. Individual steps in the application have side-effects. What's
more, these side-effects may have consequences for subsequenct
steps. In particular, a subsequent step may need to be blocked
(or performed differently) depending on the side-effect from a
As I've tried to describe, I believe that these can be done in a
(mostly) REST-ful way. And, my goal is to do it as REST-fully as
possible. Perhaps REST purists would object.
And, thanks for getting me to think about these things. I'm going
to go back and review the way in which I'm doing FSM/REST to see if
I can use some of your ideas and comments (and my replies) to
improve it, possibly making it more REST-ful.
On Tue, Apr 01, 2003 at 11:11:20AM -0700, VanL wrote:
> One final note. In his thesis, Roy Fielding attacks cookies as
> particularly breaking REST principles. However, in the redesign that
> you proposed, you note that the state is passed back and forth between
> the two actors as an XML page. How is that different in design or
> intent than a cookie that gets 'passed back and forth' representing state?
I just read the section of the thesis complaining about cookies
("126.96.36.199 Cookies"). I don't understand it too well, myself, but I
don't think it should be taken as objecting to sending content back
and forth between server and client. That, after all, is what HTTP
was intended for. Fielding's objections to cookies seem to have to
do with (1) they are hidden/opaque; (2) once set, they apply to all
future request to the same site; and (3) that they cause security
One comment from that section:
"The same functionality should have been accomplished via anonymous
authentication and true client-side state."
seems to indicate that what I'm suggesting *is* the right way,
specifically, asking the client to maintain and return state
information. But, I'd better try to find out what *true*
client-side state is.
> XML exchange
> Client sends to server XML document, representing state
> Client sends to server XML+Cookie, representing state
> Once again, I'm just not sure how the principles work.
You are right. Cookies and (XML) content are analogous, but
different. Cookies seem hidden (opaque). I'm in favor of
exchanging explicit content.
> Finally, it seems to me that there is a lot of overlap between the ideas
> of functional language design and REST web app design. The governing
> principle in both cases seems to be that the state is both passed into
> returned from the function (URI), and that there are no side-effects.
> Thus, pass in the same state+input, you get the same result -- always.
> In functional languages, you need to explicitly pass in the state of the
> application, if any state changes are required. Without a mechanism
> like cookies, though, (or XML transfer, see note above) how do you pass
> in the necessary state to accomplish the change? Finally, as I noted in
> my last email, how do you deal with the side-effects that may occur
> because of outside interactions?
> What makes web apps particularly hard is that every state that you make
> URI-accessible, you have to make a posibble starting state as well. I
> think this is the strongest of the RPCer's arguments: by having a single
> entrance point (URI), you get much better control over the execution of
> the app.
You might want to notice that in the FSM/REST model I'm suggesting,
there is a single URI for the entire FSM (but, the state changes).
----- Original Message -----
From: "Dave Kuhlman" <dkuhlman@...>
> First, I believe the claims about REST being state-less are very
> misleading. After all, the "S" in "REST" stands for "state". The
> server is state-less, but REST applications can have state.
The term 'stateless' shouldn't be applied to REST or applications build in
that style. HTTP as a protocol is called a 'stateless' protocol, because the
interpretation of any individual message is independent of any preceding
> just that maintaining state is the responsibility of the client,
> not the server. Another way to say this is to say that the server
> maintains no session information.
If your application or business logic requires the concept of 'sessions',
then model those sessions as resources & you should be fine. Think of it as
a 'working session' rather than 'protocol session ticket' or something. You
should be able to set down your 'work in progress' & pick it up later - even
from a different client.
> In the FSM/REST model that I use, state/session information is
> returned from the server to the client in an XML document. In the
> subsequent request, the client returns this XML document to the
> server. So, state information is passed back and forth between
> server and client.
If the resources involved are coarse-grained 'entry-points', then this is
more of a custom message-passing system, rather than using the built in
primitive/generic operations upon resources. The state information passed
should be a representation of a resource. The content should exclude
identification of any logical resource that the server should apply the
> I think you can see that the billing information could be passed
> back and forth between the server and client (perhaps in an XML
> document) until it is complete, error free, contains a valid credit
> card number, etc.
Yes, this sounds very reasonable. I'd be interested to know the
visible/addressable resource model involved.
> I think what we have done in this re-design, is to take what we
> might have called state, and turned it into a (temporary?)
I would approach it in the same way- but perhaps did you mean 'turn session
into a resource'.