Loading ...
Sorry, an error occurred while loading the content.
 

Re: [rest-discuss] REST & Internal Integration

Expand Messages
  • Claude Montpetit
    ... I ve been playing with both JMS and REST recently and at first, I was trying to see how REST could be used instead of JMS. I came to think that these 2
    Message 1 of 12 , May 1, 2006
      Mike Dierken a écrit :
      > Mike,
      > Sorry I haven't responded sooner... To make up for it, I wrote a really long
      > response.
      >
      > I don't have anything against JMS or messaging systems, they are great
      > tools. And just like HTTP, they work wonders when used in the right
      > situation. This thread is interesting to me because there hasn't been much
      > discussion about applying aspects of REST to a classical asynchronous
      > messaging based system.

      I've been playing with both JMS and REST recently and at first, I was
      trying to see how REST could be used instead of JMS. I came to think
      that these 2 "specifications" should not be seen as one-or-the-other.
      They are 2 different technologies that solve 2 different sets of
      problem; they can play together nicely when facilitating access to a JMS
      service is a requirement.

      In our system, we have a bunch JMS services (queue receivers and topic
      consumers). We provide custom Java client classes that clients can use
      to easily connect to the services. These talk directly to the JMS
      brokers to contact our JMS services.

      There is (well, will be) also a REST interface to the JMS services. The
      REST service allows the creation of a service request through a simple
      POST to a URI. The implementation of this REST "service" connects to the
      JMS broker, sends the corresponding message, and waits (with timeout)
      until it receives confirmation that the message was received by the JMS
      service. This confirmation of reception implies that the service request
      has created a resource and this new resource is returned to the REST
      client who can, at anytime, GET it to see the status of his orignal
      request (running, completed, cancelled, logging ...).

      So far, the JMS custom client classes are only used internally, and
      while the system has not yet been used by our clients, I expect they
      will prefer to use the REST access to the service. Even though the Java
      JMS custom classes provide finer control, REST' simplicity will most
      likely be a real advantage.

      Technically, from REST's client perspective, the launching of the
      service is synchronous: the REST server connects to the JMS service,
      waits for confirmation of reception, and returns the resource URI to the
      client. But the execution of the service request is asynchronous, and
      the REST clients can make calls on the returned URI to check the status
      of the JMS service request.



      > [...]
    • Mike Herrick
      Hi Mike, Thanks for the response. I wrote you back a missive myself ;) I hope someone other then the two of us finds this conversation useful. I d love to see
      Message 2 of 12 , May 1, 2006
        Hi Mike,

        Thanks for the response. I wrote you back a missive myself ;) I hope
        someone other then the two of us finds this conversation useful. I'd
        love to see a line drawn in the sand on whether what we are talking
        about is achievable / appropriate for REST or not. The WS-* world is
        going after stuff like this. Maybe RESTafarians will say it isn't
        appropriate, maybe it is and some impls will happen.

        You hit it on the head on why REST appeals to me when you said, "...
        payload formats independent of message framing". I look at SOAP and
        especially some of the more recent WS-* specs and see all that cruft
        right in the payload - just is awful IMHO. At first glance, the fact
        that SOAP does this seems largely immaterial ... but once you dig
        deeper you see what a negative impact it has.

        I was also thinking a few weeks ago when I read up some more on REST
        on the similarity that JMS does share with SOAP - that there are no
        application semantics as you called it. In JMS/messaging, its just a
        message. The verb is somewhere in the Destination name or somewhere
        in the XML (we hope). This is tolerable IMHO in JMS because it is a
        defined integration platform where SOAP is going over HTTP, which, is
        used for web sites and integration. I read something on SOAP vs. REST
        security & a movement a while back on forcing SOAP/HTTP to go over
        another port then 80 ... my point is at least JMS has this going for
        it. I don't know either way if I think its appropriate for JMS to
        have verbs (i.e., GET, PUT, POST, DELETE, etc.). But you are
        definitely correct that the fact that JMS does not have application
        semantics results in coordination etc. I'm very careful about this in
        my designs, but lots of people aren't.

        Sounds pretty interesting on the config - I'm cool with all config
        mechanisms that aren't JNDI (sorry I spent too many years in J2EE).
        What you describe is actually how our JMS provider works. It needs
        the config at broker start time, but caches the last good copy in
        case the management broker gets horked.

        On "stupid" services ... they are only "stupid" because they
        implement an interface that deals with all the JMS stuff. So the
        impls. really are trivial. But you are right that at the end of the
        day, developers do know a lot about JMS - they have to. My point was
        more around the details a developer doesn't have to deal with in
        terms of manually acking delivery, saving things in a temporary data
        store, etc. I'm curios on the HTTP & guaranteed delivery w/i the
        session - doesn't sound like the same thing though. With "persistent"
        messaging in JMS, the provider must write the message to disk before
        returning. This means that (unless you clown up the receiver by
        acking the message when you shouldn't), your message is going to get
        somewhere and not get lost. And if you don't ack it, it will be
        sitting on the Queue/Topic and will be resent to the receiver etc.
        This is very useful because your services don't have to account for
        this. A lot of the people I talk to who just use WS-* look at me like
        I'm really hard core for worrying about this - I think they are a
        little incompetent for not ;) When JMS is used correctly, you get
        very clear success/failures - messages either get processed correctly
        or they get sent to an error queue. You know what is going on. Not
        saying you don't w/o it, but beats the heck out of greping through
        log files. And to be fair, I've seen people screw this up with JMS so
        you don't just get it for free.

        You said, "The extra complexity of guaranteed delivery outside of a
        TCP/IP connection provides support for the disconnected use cases you
        mention, and I think this would be a great thing for a Web-scale
        system to provide, but I don't see it happening yet and I'm not sure
        why." Good to hear - so do I.

        Next you said, "My hunch is that making progress when outbound
        messages are not being delivered isn't a huge win most of the time -
        if there was work that could be accomplished without outbound network
        traffic, then it should be accomplished regardless of the messaging
        style." In my world, it is a huge win. It is a huge win when you have
        a logical transaction (e.g., submit insurance) that spans several
        services. That message could make it through 3 of the 8 services
        involved. It might make it through validation, rules engine, tiering,
        and rating, but then fail on the way to getting imaged & sent to some
        system of record. The original client (the web app where a mouse
        click happened) is long gone at this point. Lets say that for some
        reason the service that was supposed to image the insurance
        application couldn't talk to the imaging service because a router
        failed. No problem with proper JMS/messaging ... that message either
        gets routed to an error queue or "rolled back / paused" to messaging
        for future redelivery at the pause interval. Its the cumulative
        complexity of service interaction that results in messaging being
        beneficial - if you have just a couple services, you can do without
        it - its when there is coordination between the services where things
        get hairy.

        Next you said, "JMS is fine. I have to ask though, how often do you
        use JMS to integrate with groups outside your control, or outside
        your company?"

        Bingo - this is where it falls down. My company has several JMS
        providers. When certain organizations need to communicate, they need
        to bridge the providers. Not ideal obviously. This is the type of
        thing that makes me wonder if there is a better way. WS-
        ReliableMessaging is supposed to do this, but I'm not a fan of WS-
        anything. I'd love to see a reliable REST and a pub/sub - event
        driven REST if it can be done w/o hurting the web etc.

        We aren't disagreeing with each other, but you are proving my point
        when you say, "Nope, just a 50 line servlet and the messages in-
        memory in a linked list." Now, you are smart and got this right I'm
        sure - I am positive my JMS vendor did. I just freak out about
        various developers having to write their own impl. Sure you could
        have a little in house framework for this - not hard, but it is
        harder then we think to mandate etc.

        Lastly, you said: "I was kidding. The issue I was alluding to is -
        how do you know you've received all the messages or if there is still
        one more straggler? The pubsub approach is 'supposed' to be about
        loosely coupled systems, but if the publisher has to change to meet
        the specifications of each consumer, then that loose coupling has
        just tightened." Again, great point. People often do this because
        they know who is listening - they wait for the N responses (they know
        the value of N) and the return the Set of responses. Even though you
        slow things down, the only way to do this properly is to mandate a
        constant timeout (i.e., the request/reply pub/sub always takes 5
        seconds) regardless if every service who can respond and did respond
        did so in 3 seconds.

        Thanks again for the comments.

        Mike



        On Apr 29, 2006, at 11:29 PM, Mike Dierken wrote:

        > Mike,
        > Sorry I haven't responded sooner... To make up for it, I wrote a
        > really long
        > response.
        >
        > I don't have anything against JMS or messaging systems, they are great
        > tools. And just like HTTP, they work wonders when used in the right
        > situation. This thread is interesting to me because there hasn't
        > been much
        > discussion about applying aspects of REST to a classical asynchronous
        > messaging based system.
        > In my opinion, there are a couple things that are similar between
        > the two -
        > simple and common interactions for many kinds of applications (send
        > message,
        > receive message, etc), payload formats independent of message
        > framing, and
        > so on. But there are a few things common in REST that I don't see
        > in JMS or
        > other messaging systems :
        >
        > 1) application semantics : sending a message in REST means something
        > (delete, modify, retrieve, etc), whereas sending a message in JMS
        > or other
        > messaging systems is just sending a message (and it usually has
        > unknown
        > consequences, which requires endless coordination to work out the
        > details)
        >
        > 2) resource identifiers : this sort of follows from being an
        > application
        > protocol - the protocol is all about interacting with resources, which
        > provides a framework for modelling applications (above just moving
        > bytes
        > from point a to point b).
        >
        > 3) hypertext (as the engine of application state) : this is one of the
        > facets of REST that took me the longest to put into practice, but
        > has paid
        > off very well. Using documents with tags that help pick out the
        > resource
        > identifiers is invaluable in a dynamic, adaptive and configuration-
        > free
        > system. Many systems have dynamically retrieved configurations, and
        > this
        > provides part of the benefit of hypertext, and when links (resource
        > identifiers that can be interacted with) are used a lot of config file
        > processing becomes generalizable and just falls away. For example,
        > at work
        > we recently built a large scale real-time data publishing system
        > and the
        > consuming systems have a link to a 'directory service', which is an
        > xml blob
        > with elements with links to the subjects to subscribe to, links to the
        > cold-start data to bootstrap with and so on. The cold-start data is
        > somewhat
        > large, but by teaching the clients to 'just follow the links', the
        > data
        > could be arbitrarily partitioned by the server. In addition, with a
        > little
        > thought you could put multiple links in there and have parallel
        > streams
        > being retrieved by the client - a good way to utilize all that
        > bandwidth
        > just laying around.
        >
        >
        >> I spend most of my time working on a system that has approximately 50
        >> services. The services are pretty stupid, all they know is
        >> they get an event XML input. They typically respond with an
        >> XML event. They listen on a set of JMS Topics or a Topic
        >> pattern (e.g., COMPANYNAME.LINEOFBUSINESS.NOUN.*).
        >> They also typically respond to set of Topics. What I like
        >> about JMS in the system is the guaranteed delivery & durable
        >> subscriptions. The service impls don't care about the network
        >> because they know that JMS is there.
        > These are extremely useful tools - durable subscriptions, guaranteed
        > delivery, etc. But from my point of view the 'service' isn't all that
        > 'stupid'. The service is everything the outside world would see,
        > and that
        > includes those handy things like disconnected use and reliability.
        > The code
        > that plugs into this is simplified by that environment, which is
        > hufely
        > valuable. I'm not a messaging or protocol expert, but I always
        > thought that
        > HTTP over TCP/IP provided guaranteed delivery within the connection
        > - I'd
        > like to hear from others on how we should consider the 'guarantee-
        > ness' of
        > HTTP (within a connection). The extra complexity of guaranteed
        > delivery
        > outside of a TCP/IP connection provides support for the
        > disconnected use
        > cases you mention, and I think this would be a great thing for a
        > Web-scale
        > system to provide, but I don't see it happening yet and I'm not
        > sure why.
        >
        >
        >> Also, durable subscriptions are great [...]
        >> I would really like to see an HTTP based pub/sub like this.
        >> Idon't know enough about REST to know if it would be evil to do.
        >> I babbled about that a bit more here:
        >> http://fuzzypanic.blogspot.com/2006/03/httppox-xml-based-eda-pub-
        >> sub.html
        >> Let me know if you know of anything.
        > There have been a few companies doing that - you found mod_pubsub
        > already,
        > which was a spin-out from KnowNow (which I helped start up many
        > years ago,
        > for a short period of time). Supposedly Renkoo may be doing work to
        > make
        > mod-pubsub live again. The toy 'messaging' web app I linked to last
        > time
        > shows that it's very easy to build messaging with pure HTTP, and
        > with modern
        > systems supporting non-blocking IO, it's not too hard to be
        > efficient. The
        > hard part is the pump that keeps the messages flowing along the
        > durable
        > subscriptions, dealing with cycles and other plumbing issues.
        >
        > You ask a very good question though - is pubsub compatible with
        > REST? If
        > REST is about application level semantics, you could model messages,
        > subscriptions, etc. as resources and go that route. But then what
        > are the
        > messages about? How are those raised to the application level so
        > that common
        > semantics can be applied?
        >
        >
        >> Sure, but that block means something typically - it means the
        >> message was persisted so the client doesn't have to ferret it
        >> away just in case something bad happens later.
        >> With Web Style, you can do that too - my point is you have to do that
        >> yourself (client ferret away the data that was PUT/POST).
        > I see this as partly a difference in whether the client host or the
        > server
        > host has some persistent storage. If there is no client persistent
        > storage,
        > then the client application logic needs to know when the server has
        > taken
        > hold of the data in some persistent fashion (if that's the goal in
        > that
        > situation). The other part is how well the application behaves in
        > the face
        > of an outage or disconnection - picking up where you left off is
        > harder if
        > the application doesn't store the state as it makes progress. My
        > hunch is
        > that making progress when outbound messages are not being delivered
        > isn't a
        > huge win most of the time - if there was work that could be
        > accomplished
        > without outbound network traffic, then it should be accomplished
        > regardless
        > of the messaging style.
        >
        >
        >> Nope, I mean make pub/sub more RESTful.
        >> Like I said, I don't know enough about REST principals to know if
        >> this is evil or not. I have a decent messaging middleware product
        >> today.
        >> I'm happy with it - I just want to know if its overkill or not or
        >> if there is a simpler way. If some of my integration types
        >> are best handled via JMS, fine with me. I just want to ask
        >> the "stupid question" - to make sure I'm not thinking JMS is
        >> better then it is.
        > JMS is fine. I have to ask though, how often do you use JMS to
        > integrate
        > with groups outside your control, or outside your company?
        >
        >>> Here is a sample app I wrote a while back that does what you are
        >>> talking
        > about :
        >>>
        > http://www.searchalert.net:8080/dierken/events/chat.html?topic=rest-
        > discuss
        >>> The underlying system holds messages here:
        > http://www.searchalert.net:8080/xmlrouter/msgs/rest-discuss
        >> So what is on the server - is there messaging behind it?
        > Nope, just a 50 line servlet and the messages in-memory in a linked
        > list.
        >
        >
        >
        >>> Are you sure you didn't mean N+1 responses?
        >> I'm talking about a client sending 1 request, having N
        >> services subscribe and send a response to the client.
        > I was kidding. The issue I was alluding to is - how do you know you've
        > received all the messages or if there is still one more straggler? The
        > pubsub approach is 'supposed' to be about loosely coupled systems,
        > but if
        > the publisher has to change to meet the specifications of each
        > consumer,
        > then that loose coupling has just tightened.
        >
      • Mike Dierken
        ... I did something like that about five years ago : http://sourceforge.net/projects/destiny That project is more of a (working) demo but the ideas are valid &
        Message 3 of 12 , May 1, 2006
          >
          > There is (well, will be) also a REST interface to the JMS
          > services. The REST service allows the creation of a service
          > request through a simple POST to a URI. The implementation of
          > this REST "service" connects to the JMS broker, sends the
          > corresponding message, and waits (with timeout) until it
          > receives confirmation that the message was received by the
          > JMS service. This confirmation of reception implies that the
          > service request has created a resource and this new resource
          > is returned to the REST client who can, at anytime, GET it to
          > see the status of his orignal request (running, completed,
          > cancelled, logging ...).
          I did something like that about five years ago :
          http://sourceforge.net/projects/destiny
          That project is more of a (working) demo but the ideas are valid & can be
          very useful.
        Your message has been successfully submitted and would be delivered to recipients shortly.