Loading ...
Sorry, an error occurred while loading the content.

Re: [rest-discuss] Atompub missed opportunity

Expand Messages
  • Benjamin Carlyle
    ... Exactly. It has been agreed ahead of time. Now, I can just send my POST request and know that the server understands a useful subset of what I am sending
    Message 1 of 25 , Dec 31, 2006
    • 0 Attachment
      On Sat, 2006-12-30 at 19:08 -0600, Hugh Winkler wrote:
      > On 12/30/06, Benjamin Carlyle <benjamincarlyle@...> wrote:
      > > I'm not sure who you think will interpret this form on the client
      > side.
      > > You see, the thing is that the atom document format is already a
      > form. A
      > > client has already received the form by being programmed in a
      > particular
      > > way, and is now submitting the form. It knows what information
      > should be
      > > placed in each of the named fields. It knows how to construct the
      > end
      > > document.
      > Ah, but it is a standard form for all servers -- not for my particular
      > server app. And to know what kind of form to submit, you read a spec
      > -- you did not GET the description from the server. It's all baked in
      > at design time.

      Exactly. It has been agreed ahead of time. Now, I can just send my POST
      request and know that the server understands a useful subset of what I
      am sending them.

      What you are suggesting is that I first need to obtain a schema document
      (which you are calling a form) to see if the server is actually
      understanding only a subset of the atom vocabulary. Then I need to
      customize my content to conform to this subset. As a machine, I don't
      have any good way of doing that. If I come up to a server that has
      support for an unexpectedly small subset of atom, I then have to
      customize my content in an unexpected way. That is to say, I way that
      noone programmed me to customise my content in :)

      The server, on the other hand, is in a good position to customise the
      content. It knows which subset of atom it understands, and it knows what
      atom is generally. It knows multiple authors might be required, so is in
      a position to either model those multiple authors or use an algorithm to
      select an author for its model from the available list.

      > What I proposed is that the form delivered by a server have just the
      > elements that make sense for the server. My server might not know what
      > to do with a <source> element. Using the standard "form", my server
      > has to have handling in place if you submit a <source> element, and it
      > has to describe to you the problem if you do submit <source> and my
      > server rejects it.

      All atom elements make sense to the server, even if they don't fit the
      server's internal model. The server implements atom, after all. What you
      are suggesting is adding an extra message exchange to move the
      complexity of fitting atom into the server's model back to the client.
      Neither side is going to be great at solving a model mismatch problem,
      but the client is likely to be downright incapable. If you really want
      to be able to sensibly deal with this class of server you need to encode
      how that class of server behaves into the atom specifications (as you
      suggested was already being done). Then the server will use a
      generally-accepted algorithm to select which author it will use from the
      ones available, and the client will know to put the most important
      author first.

      > > One might argue that the client and server should communicate online
      > > before the client submits data that the server might not accept. In
      > one
      > > of the examples you mentioned earlier the server might not support
      > the
      > > full atom protocol, in that multiple authors may not be supported.
      > To be clear -- whether a server has a model supporting multiple
      > authors is a modeling, not a message exchange protocol, issue.

      The server's model doesn't allow for a full fidelity translation of the
      atom protocol. The model doesn't allow the full protocol to be
      understood.

      > > As
      > > such, the server might be able to publish a document that indicated
      > a
      > > subset of the atom format that is usable.
      > >I suggest that while a
      > > document format could be produced to communicate which subset is
      > > important, the cost would likely outweigh the value. Certainly,
      > allowing
      > > for additional elements in the form would not be meaningful to a
      > client
      > > without further specificaiton as to what those elements mean. Any
      > > additions of this kind would likely require code changes to the
      > client.
      >
      > Well I was focusing on what clients POST to servers, but it seemed
      > like you began that para talking about what servers publish?
      >
      > In the case of servers describing what clients can send to them, that
      > would just be the Xform, so I'm not clear how the cost outweighs the
      > value -- it seems simple enough in this case.

      Hrrm... why not xml schema? Why not relax ng? Why not another format?

      Naming a particular specification doesn't really help. A standardisation
      process still has to take place to achieve widespread acceptance.

      Your server can still offer an xform or a HTML form to allow new atom
      entries to be created by humans. That needs no further standardisation
      to occur. However, machine to machine communications is best and
      simplest when the specification can be treated as the form that needs to
      be filled out and understood. Additional negotiation between client and
      server reduces the value of the standard and increases complexity
      everwhere.

      Benjamin.
    • Steve G. Bjorg
      ... To do input validation. Let me explain: I m currently pondering how to best deal with resources that are pure behaviors (i.e. only accept POST) in Dream,
      Message 2 of 25 , Jan 1, 2007
      • 0 Attachment
        Hugh Winkler wrote:
        > >
        > > In other words, I can see how the form can be generated from the
        > > schema (modulo input field descriptions), but I'm not sure how to
        > > infer the complete schema from a form. Are there examples of this?
        > >
        >
        > Unsure I'm understanding you: Why would you want to infer a complete
        > schema from a form?

        To do input validation. Let me explain: I'm currently pondering how
        to best deal with resources that are pure behaviors (i.e. only accept
        POST) in Dream, our REST framework. If I associate a RelaxNG schema
        with the resource, I can use it to validate input before it reaches
        the implementation; and I _could_ use it to generate a form. The form
        would be returned on a GET request, making the resource reflective (or
        self-descriptive). Now, developers could use the form to interact
        with the resource directly, which has a lot pedagogical value.

        However, if I take the opposite approach, meaning that if a GET were
        to return a form, I would not get the automatic input validation
        benefit unless I were able to generate the schema from the form.

        Thus, my current inclination is to use a RelaxNG schema to both do
        input validation and interact with the resource.


        - Steve

        --------------
        Steve G. Bjorg
        http://www.mindtouch.com
        http://www.opengarden.org
      • Hugh Winkler
        ... That s the problem --- ahead of time rather than dynamically at run time. ... Not sure how you would know it understands a useful subset. You won t have
        Message 3 of 25 , Jan 2, 2007
        • 0 Attachment
          On 12/31/06, Benjamin Carlyle <benjamincarlyle@...> wrote:
          > On Sat, 2006-12-30 at 19:08 -0600, Hugh Winkler wrote:
          > > On 12/30/06, Benjamin Carlyle <benjamincarlyle@...> wrote:
          > > > I'm not sure who you think will interpret this form on the client
          > > side.
          > > > You see, the thing is that the atom document format is already a
          > > form. A
          > > > client has already received the form by being programmed in a
          > > particular
          > > > way, and is now submitting the form. It knows what information
          > > should be
          > > > placed in each of the named fields. It knows how to construct the
          > > end
          > > > document.
          > > Ah, but it is a standard form for all servers -- not for my particular
          > > server app. And to know what kind of form to submit, you read a spec
          > > -- you did not GET the description from the server. It's all baked in
          > > at design time.
          >
          > Exactly. It has been agreed ahead of time.

          That's the problem --- "ahead of time" rather than dynamically at run time.

          >Now, I can just send my POST
          > request and know that the server understands a useful subset of what I
          > am sending them.

          Not sure how you would know it understands a useful subset. You won't
          have the slightest idea what parts of your document it understands, or
          doesn't. doesn't handle multiple authors? Can only accept text/plain
          for <title>?

          >
          > What you are suggesting is that I first need to obtain a schema document
          > (which you are calling a form)

          A schema document is not a form. Maybe we should stay away from XML
          for a moment. Think HTML form... which is a little "program" telling
          the UA how to serialize a submission.

          I didn't mention RDF forms[1]... but I should have.

          > to see if the server is actually
          > understanding only a subset of the atom vocabulary.

          Not a subset... could be a superset.

          > Then I need to
          > customize my content to conform to this subset. As a machine, I don't
          > have any good way of doing that.

          See, you have this problem anyway. If you are sending a server stuff
          it doesn't understand, or not enough stuff, your application will
          fail.

          With forms, at least, your application knows "Hey, I don't know how to
          fill out this required field". Same as a human would using an HTML
          form. Your client can report that to a human for correction.

          Atom (and any application protocol based on exchanging known document
          types) has to trade off between exhaustively specifying application
          behavior and exhaustively specifying failure handling.

          >If I come up to a server that has
          > support for an unexpectedly small subset of atom, I then have to
          > customize my content in an unexpected way.

          It is better for you to do the customizing. TTake the example of a
          server that simply cannot honor text/html or application/xhtml+xml in
          the title field. It can only handle text/plain. Atom protocol says
          nothing at the moment about this situation, except that the server can
          change your POSted data as it needs to. So presently my server either
          a) rejects your submission or b) stores it as text/plain. Better
          would be for your client to receive an Xform with a constraint
          specifiying "text/plain" only -- then, if the user had any imprtant
          rich content they wanted to put in the title, they can at least try to
          compensate.


          >That is to say, I way that
          > noone programmed me to customise my content in :)
          >
          Well, you would have programmed it from this pov, so you would have
          handled these exceptional situations.

          > The server, on the other hand, is in a good position to customise the
          > content. It knows which subset of atom it understands, and it knows what
          > atom is generally. It knows multiple authors might be required, so is in
          > a position to either model those multiple authors or use an algorithm to
          > select an author for its model from the available list.


          See above. Yes, you are describing the undesirable behavior the
          current APP forces you into.

          >
          > > What I proposed is that the form delivered by a server have just the
          > > elements that make sense for the server. My server might not know what
          > > to do with a <source> element. Using the standard "form", my server
          > > has to have handling in place if you submit a <source> element, and it
          > > has to describe to you the problem if you do submit <source> and my
          > > server rejects it.
          >
          > All atom elements make sense to the server, even if they don't fit the
          > server's internal model.
          > The server implements atom, after all.

          not so -- the client may submit extgension elements the server isn't aware of.

          > What you
          > are suggesting is adding an extra message exchange to move the
          > complexity of fitting atom into the server's model back to the client.

          Yes.

          > Neither side is going to be great at solving a model mismatch problem,
          > but the client is likely to be downright incapable.

          As above: The client is in the best position to take corrective
          action, so as to best fulfill its intent.

          >If you really want
          > to be able to sensibly deal with this class of server you need to encode
          > how that class of server behaves into the atom specifications (as you
          > suggested was already being done). Then the server will use a
          > generally-accepted algorithm to select which author it will use from the
          > ones available, and the client will know to put the most important
          > author first.
          >

          Yes... app is prescribing server behavior, not message semantics.

          > > > One might argue that the client and server should communicate online
          > > > before the client submits data that the server might not accept. In
          > > one
          > > > of the examples you mentioned earlier the server might not support
          > > the
          > > > full atom protocol, in that multiple authors may not be supported.
          > > To be clear -- whether a server has a model supporting multiple
          > > authors is a modeling, not a message exchange protocol, issue.
          >
          > The server's model doesn't allow for a full fidelity translation of the
          > atom protocol. The model doesn't allow the full protocol to be
          > understood.
          >
          > > > As
          > > > such, the server might be able to publish a document that indicated
          > > a
          > > > subset of the atom format that is usable.
          > > >I suggest that while a
          > > > document format could be produced to communicate which subset is
          > > > important, the cost would likely outweigh the value. Certainly,
          > > allowing
          > > > for additional elements in the form would not be meaningful to a
          > > client
          > > > without further specificaiton as to what those elements mean. Any
          > > > additions of this kind would likely require code changes to the
          > > client.
          > >
          > > Well I was focusing on what clients POST to servers, but it seemed
          > > like you began that para talking about what servers publish?
          > >
          > > In the case of servers describing what clients can send to them, that
          > > would just be the Xform, so I'm not clear how the cost outweighs the
          > > value -- it seems simple enough in this case.
          >
          > Hrrm... why not xml schema? Why not relax ng? Why not another format?
          >

          Schema != Form, but yeah, another kind of form ... RDF form...


          > Naming a particular specification doesn't really help. A standardisation
          > process still has to take place to achieve widespread acceptance.
          >

          It's already in place. Programs understand the meanings of Atom XML
          elements. So when presented with an Xform model specifying those
          elements, the client app kknows how to populate those fields.

          > Your server can still offer an xform or a HTML form to allow new atom
          > entries to be created by humans. That needs no further standardisation
          > to occur. However, machine to machine communications is best and
          > simplest when the specification can be treated as the form that needs to
          > be filled out and understood.

          Well heck... if you believe that, then RPC is a good approach... it's
          all spelled out for the client programmers at design time.

          >Additional negotiation between client and
          > server reduces the value of the standard and increases complexity
          > everwhere.
          >

          Forms made the web adaptive. Go to any airline reservation web site.
          They're all the same, but different too. They mostly do the same
          things, but Orbitz offers packages with hotels and cars, while Delta
          offers connections with partner airlines, and Priceline won't let you
          see the itinerary. There's some shared vocabulary among all those
          sites but varying behavior. If the airlines standardize their
          vocabulary for forms, you could program a client to interact,
          adaptively, with all those sites. But you could not constrain
          Priceline's app to squeeze into the same behavior model as Delta's.
          And you shouldn't -- you should encourage diversity among web apps, as
          has been successful on the web to date.


          Hugh
          [1] http://www.markbaker.ca/2003/05/RDF-Forms/
        • Benjamin Carlyle
          ... What you have to understand here, is that you are trying to replace human agreement with machine agreement. Humans are good at negotiating standards like
          Message 4 of 25 , Jan 3, 2007
          • 0 Attachment
            On Tue, 2007-01-02 at 12:31 -0600, Hugh Winkler wrote:
            > On 12/31/06, Benjamin Carlyle <benjamincarlyle@...> wrote:
            > > On Sat, 2006-12-30 at 19:08 -0600, Hugh Winkler wrote:
            > > > On 12/30/06, Benjamin Carlyle <benjamincarlyle@...> wrote:
            > > > > I'm not sure who you think will interpret this form on the client
            > > > side.
            > > > > You see, the thing is that the atom document format is already a
            > > > form. A
            > > > > client has already received the form by being programmed in a
            > > > particular
            > > > > way, and is now submitting the form. It knows what information
            > > > should be
            > > > > placed in each of the named fields. It knows how to construct the
            > > > end
            > > > > document.
            > > > Ah, but it is a standard form for all servers -- not for my particular
            > > > server app. And to know what kind of form to submit, you read a spec
            > > > -- you did not GET the description from the server. It's all baked in
            > > > at design time.
            > > Exactly. It has been agreed ahead of time.
            > That's the problem --- "ahead of time" rather than dynamically at run time.

            What you have to understand here, is that you are trying to replace
            human agreement with machine agreement. Humans are good at negotiating
            standards like atom. It's a hard problem, but standards get nailed out.
            Information producers agree to transform their internal models into the
            standard format. Information consumers agree that the standard format is
            a suitable source of data for their internal models. Internal models
            often have to bend as part of this process, and eventually realign
            around a competent standard to be more simliar than they are different.

            Machines are not good at negotiation. You give a machine a form to fill
            out, and the machine already needs to know how to fill out the form
            before it starts. The form says "title", "summary", "content". The
            machine already needs an internal model that has those elements. The
            form says "don't give me the summary", the client could have code
            written to say "only send the subset of the standard which the server
            says it can accept". That's as good as you can do.

            You can't give a client an arbitrary form that isn't a simple subset and
            expect it to know what to do. If the client software wasn't written to
            know that only one author might be supported on the other side, it can't
            choose which author it should supply any better than the server side.
            You are lucky if it can interpret the server's instructions not to
            supply more than one at all.

            You can't arbitrarily place restrictions on the client as to how it
            should fill out its content. The only practical way to do it is to write
            a program that the client must run over a standard atom data model in
            order to fit the server's point of view. And guess what: That's the same
            program you would run on the server side if the client just submitted
            the atom document in the first place.

            Clients can't deal with unexpected server demands. Server demands are
            only expected if they are negotiated between humans, which is to say
            they are part of the atom specification. You can't do any better than
            what is in the standard by supplying a form.

            > >Now, I can just send my POST
            > > request and know that the server understands a useful subset of what I
            > > am sending them.
            > Not sure how you would know it understands a useful subset. You won't
            > have the slightest idea what parts of your document it understands, or
            > doesn't. doesn't handle multiple authors? Can only accept text/plain
            > for <title>?

            I know it understands because we agreed on the content through the atom
            standardisation process. We agreed that I would send this much and the
            server would understand that much. Whether understand means "completely
            model" is up to the server. It is free to cut the xhtml out of its
            title. It is even free to use the xhtml content as text/plain. That's
            its perrogative. What is not in its perrogative is to reject a
            well-formed and valid atom document. If it intends to do that it should
            not claim to understand atom in the first place.

            > > What you are suggesting is that I first need to obtain a schema document
            > > (which you are calling a form)
            > A schema document is not a form. Maybe we should stay away from XML
            > for a moment. Think HTML form... which is a little "program" telling
            > the UA how to serialize a submission.

            You are describing the set of valid documents I can submit to you. You
            can call it a form if you like, but it is more correctly a schema.

            If you are no longer talking about a schema, and are now talking about a
            program to transform my atom content into your sub-atom content... then
            why aren't you running that program on the server side?

            > > to see if the server is actually
            > > understanding only a subset of the atom vocabulary.
            > Not a subset... could be a superset.

            Now you are talking about the client supplying more elements than it
            knows how to supply. You are presumably talking about extensions to the
            standard, but extensions are standards too. Extensions require human
            agreement between client and server in order to be understood.

            > > Then I need to
            > > customize my content to conform to this subset. As a machine, I don't
            > > have any good way of doing that.
            > See, you have this problem anyway. If you are sending a server stuff
            > it doesn't understand, or not enough stuff, your application will
            > fail.

            But I have already agreed through the standardisation process with the
            server that it will understand my content. My application will only fail
            if the server fails to implement the specification.

            > With forms, at least, your application knows "Hey, I don't know how to
            > fill out this required field". Same as a human would using an HTML
            > form. Your client can report that to a human for correction.

            It only knows if I write code. I only write code if I have communicated
            with the guy who wrote the server about what is permissible. I have
            already done this. We called that conversation the atom standardisation
            process.

            Why do you think a human in the loop can do anything about the failure
            to communicate? Are they going to hack on their client application every
            time a server says it only understands an unexpectedly-small subset of
            atom or demands an extension element be supplied? No... if there is a
            human in the loop she will write an email to the server's administrator
            to inform him of his bug in failing to implement the specification. It
            is not the client's problem. It is the server's problem.

            > Atom (and any application protocol based on exchanging known document
            > types) has to trade off between exhaustively specifying application
            > behavior and exhaustively specifying failure handling.

            And exhaustively supporting forwards-compatability for extensions, and
            exhaustively trading server and client-side complexity for protocol
            features.

            > >If I come up to a server that has
            > > support for an unexpectedly small subset of atom, I then have to
            > > customize my content in an unexpected way.
            > It is better for you to do the customizing. TTake the example of a
            > server that simply cannot honor text/html or application/xhtml+xml in
            > the title field. It can only handle text/plain. Atom protocol says
            > nothing at the moment about this situation, except that the server can
            > change your POSted data as it needs to. So presently my server either
            > a) rejects your submission or b) stores it as text/plain. Better
            > would be for your client to receive an Xform with a constraint
            > specifiying "text/plain" only -- then, if the user had any imprtant
            > rich content they wanted to put in the title, they can at least try to
            > compensate.

            No no no.... the client doesn't know how to customise the content. As
            the author of the client I relied on the atom specification that says I
            can supply a content element and I did. Now your server is telling me it
            doesn't understand it and wants a summary element instead? My client is
            not written to deal with that. The server can deal with its own
            shortcomings, thankyou. If it doesn't understand the protocol it should
            stop speaking it.

            If the server doesn't understand xhtml in the title, then tough! It
            knows that it must be expected to deal with xhtml in the title because
            we agreed through the standardisation process that it should be capable.
            How it provides that support as the atom spec rightly points out is up
            to the server. Maybe it will strip out anything in angle-brackets before
            storing the value into its internal title variable. Maybe it will just
            use the xhtml content verbatim. That's the server's problem. It isn't
            permitted to reject my submission. How is my client supposed to know
            that what it needs to do in the face of this dumb server? We already
            agreed that xhtml was fine, and now this server wants to go back on
            that? Move complexity to me, will you? No thanks. I'll find another
            server to talk to. Maybe one that understands xhtml in the title.

            > >That is to say, I way that
            > > noone programmed me to customise my content in :)
            > Well, you would have programmed it from this pov, so you would have
            > handled these exceptional situations.

            I don't write client software to deal with broken servers that don't
            implment the spec. It is up to the server to deal with the problem if it
            can't translate my request precisely.

            > > The server, on the other hand, is in a good position to customise the
            > > content. It knows which subset of atom it understands, and it knows what
            > > atom is generally. It knows multiple authors might be required, so is in
            > > a position to either model those multiple authors or use an algorithm to
            > > select an author for its model from the available list.
            > See above. Yes, you are describing the undesirable behavior the
            > current APP forces you into.

            Quite the opposite. Your suggestion doesn't hold water. The spec
            reflects decades of experience in developing protocols that work and can
            evolve succesfully for decades to come. Do you think you can do better
            without having written software on more than one side of the
            client/server fence?

            > > > What I proposed is that the form delivered by a server have just the
            > > > elements that make sense for the server. My server might not know what
            > > > to do with a <source> element. Using the standard "form", my server
            > > > has to have handling in place if you submit a <source> element, and it
            > > > has to describe to you the problem if you do submit <source> and my
            > > > server rejects it.
            > > All atom elements make sense to the server, even if they don't fit the
            > > server's internal model.
            > > The server implements atom, after all.
            > not so -- the client may submit extgension elements the server isn't aware of.

            And the server is required to ignore them and the client is required to
            accept that old servers will ignore them. That's the way extensible
            protocols work. If the extension is good it will be supported. If it is
            not it will be ignored, sidelined, and eventually forgotten.

            > > What you
            > > are suggesting is adding an extra message exchange to move the
            > > complexity of fitting atom into the server's model back to the client.
            > > Neither side is going to be great at solving a model mismatch problem,
            > > but the client is likely to be downright incapable.
            > As above: The client is in the best position to take corrective
            > action, so as to best fulfill its intent.

            Show me the code. Your solution requires me to write client-side code
            every time a new kind of dumb server is placed on the internet. That
            doesn't work. That doesn't scale. Machines can't negotiate, only humans
            can... and we already have. If you have anything more to add to that
            conversation you had better to do it. Don't try to hold a separate
            conversation with my client software. It doesn't know how to hold that
            conversation.

            > Forms made the web adaptive. Go to any airline reservation web site.
            > They're all the same, but different too. They mostly do the same
            > things, but Orbitz offers packages with hotels and cars, while Delta
            > offers connections with partner airlines, and Priceline won't let you
            > see the itinerary. There's some shared vocabulary among all those
            > sites but varying behavior. If the airlines standardize their
            > vocabulary for forms, you could program a client to interact,
            > adaptively, with all those sites. But you could not constrain
            > Priceline's app to squeeze into the same behavior model as Delta's.
            > And you shouldn't -- you should encourage diversity among web apps, as
            > has been successful on the web to date.

            The evolvability of HTML and HTTP are what have been successful on the
            web to date. Atom follows the evolvability and agreement model of its
            predecessors. Do you really want your web server to have to retrieve a
            form from every browser that requests a html page before it can be
            returned? When that form says "I don't understand paragraph markers",
            what will your web server do to make the content fit?

            Benjamin.
          • Hugh Winkler
            Benjamin, ... Objection: Clients can t be programmed to complete forms Counter-example: This machine automatically fills out a form based on a vocabulary
            Message 5 of 25 , Jan 3, 2007
            • 0 Attachment
              Benjamin,

              ----------------------
              Objection: "Clients can't be programmed to complete forms"

              Counter-example: This machine automatically fills out a form based on
              a vocabulary every day: Your web browser, when it pre-fills form
              fields for you. Browsers know that user name and password fields on
              HTML forms usually have consistent names. And Google Toolbar saves
              some common personal info about you and auto-populates your name,
              address, etc, based on a common vocabulary. You just program a client
              to map information it has, to fields identified by name -- a
              vocabulary.


              ----------------------
              Misconception: "Forms are schemas"

              Correction: Forms are queries made by the server to the client. "Give
              me the data you have named 'author-email". Schemas describe relations
              among objects. They describe models, not messages.


              ----------------------

              Starry-eyed delusion: A Standards process can capture the union of all
              the semantics any server could manifest.
              Corollary: Exchanging these documents is "scalable"

              Real world experience: RosettaNet PIP 3A4. Thoroughly specified
              schemas describing almost every possible Purchase Order Request and
              Response. Guess how long it takes one large computer manufacturer to
              shake out enough impedance mismatches to get going with each new
              trading partner? 30-60 days, and that is a dedicated team of experts
              exchanging test messages, "validating" them, discovering and resolving
              the semantic mismatches in even the "valid" documents. So that's one
              or two months per partner. How long does it take to ramp on just 100
              partners? Years. Not web scale, is it?

              Exchanging pre-defined special purpose documents, as APP does,
              satisfies REST's definition, but it's not webby. Forms have proven
              their scalability on the web, because they promote loose coupling.
              They offer hope that web services could be deployed at web scale.

              Hugh


              On 1/3/07, Benjamin Carlyle <benjamincarlyle@...> wrote:
              > On Tue, 2007-01-02 at 12:31 -0600, Hugh Winkler wrote:
              > > On 12/31/06, Benjamin Carlyle <benjamincarlyle@...> wrote:
              > > > On Sat, 2006-12-30 at 19:08 -0600, Hugh Winkler wrote:
              > > > > On 12/30/06, Benjamin Carlyle <benjamincarlyle@...> wrote:
              > > > > > I'm not sure who you think will interpret this form on the client
              > > > > side.
              > > > > > You see, the thing is that the atom document format is already a
              > > > > form. A
              > > > > > client has already received the form by being programmed in a
              > > > > particular
              > > > > > way, and is now submitting the form. It knows what information
              > > > > should be
              > > > > > placed in each of the named fields. It knows how to construct the
              > > > > end
              > > > > > document.
              > > > > Ah, but it is a standard form for all servers -- not for my particular
              > > > > server app. And to know what kind of form to submit, you read a spec
              > > > > -- you did not GET the description from the server. It's all baked in
              > > > > at design time.
              > > > Exactly. It has been agreed ahead of time.
              > > That's the problem --- "ahead of time" rather than dynamically at run time.
              >
              > What you have to understand here, is that you are trying to replace
              > human agreement with machine agreement. Humans are good at negotiating
              > standards like atom. It's a hard problem, but standards get nailed out.
              > Information producers agree to transform their internal models into the
              > standard format. Information consumers agree that the standard format is
              > a suitable source of data for their internal models. Internal models
              > often have to bend as part of this process, and eventually realign
              > around a competent standard to be more simliar than they are different.
              >
              > Machines are not good at negotiation. You give a machine a form to fill
              > out, and the machine already needs to know how to fill out the form
              > before it starts. The form says "title", "summary", "content". The
              > machine already needs an internal model that has those elements. The
              > form says "don't give me the summary", the client could have code
              > written to say "only send the subset of the standard which the server
              > says it can accept". That's as good as you can do.
              >
              > You can't give a client an arbitrary form that isn't a simple subset and
              > expect it to know what to do. If the client software wasn't written to
              > know that only one author might be supported on the other side, it can't
              > choose which author it should supply any better than the server side.
              > You are lucky if it can interpret the server's instructions not to
              > supply more than one at all.
              >
              > You can't arbitrarily place restrictions on the client as to how it
              > should fill out its content. The only practical way to do it is to write
              > a program that the client must run over a standard atom data model in
              > order to fit the server's point of view. And guess what: That's the same
              > program you would run on the server side if the client just submitted
              > the atom document in the first place.
              >
              > Clients can't deal with unexpected server demands. Server demands are
              > only expected if they are negotiated between humans, which is to say
              > they are part of the atom specification. You can't do any better than
              > what is in the standard by supplying a form.
              >
              > > >Now, I can just send my POST
              > > > request and know that the server understands a useful subset of what I
              > > > am sending them.
              > > Not sure how you would know it understands a useful subset. You won't
              > > have the slightest idea what parts of your document it understands, or
              > > doesn't. doesn't handle multiple authors? Can only accept text/plain
              > > for <title>?
              >
              > I know it understands because we agreed on the content through the atom
              > standardisation process. We agreed that I would send this much and the
              > server would understand that much. Whether understand means "completely
              > model" is up to the server. It is free to cut the xhtml out of its
              > title. It is even free to use the xhtml content as text/plain. That's
              > its perrogative. What is not in its perrogative is to reject a
              > well-formed and valid atom document. If it intends to do that it should
              > not claim to understand atom in the first place.
              >
              > > > What you are suggesting is that I first need to obtain a schema document
              > > > (which you are calling a form)
              > > A schema document is not a form. Maybe we should stay away from XML
              > > for a moment. Think HTML form... which is a little "program" telling
              > > the UA how to serialize a submission.
              >
              > You are describing the set of valid documents I can submit to you. You
              > can call it a form if you like, but it is more correctly a schema.
              >
              > If you are no longer talking about a schema, and are now talking about a
              > program to transform my atom content into your sub-atom content... then
              > why aren't you running that program on the server side?
              >
              > > > to see if the server is actually
              > > > understanding only a subset of the atom vocabulary.
              > > Not a subset... could be a superset.
              >
              > Now you are talking about the client supplying more elements than it
              > knows how to supply. You are presumably talking about extensions to the
              > standard, but extensions are standards too. Extensions require human
              > agreement between client and server in order to be understood.
              >
              > > > Then I need to
              > > > customize my content to conform to this subset. As a machine, I don't
              > > > have any good way of doing that.
              > > See, you have this problem anyway. If you are sending a server stuff
              > > it doesn't understand, or not enough stuff, your application will
              > > fail.
              >
              > But I have already agreed through the standardisation process with the
              > server that it will understand my content. My application will only fail
              > if the server fails to implement the specification.
              >
              > > With forms, at least, your application knows "Hey, I don't know how to
              > > fill out this required field". Same as a human would using an HTML
              > > form. Your client can report that to a human for correction.
              >
              > It only knows if I write code. I only write code if I have communicated
              > with the guy who wrote the server about what is permissible. I have
              > already done this. We called that conversation the atom standardisation
              > process.
              >
              > Why do you think a human in the loop can do anything about the failure
              > to communicate? Are they going to hack on their client application every
              > time a server says it only understands an unexpectedly-small subset of
              > atom or demands an extension element be supplied? No... if there is a
              > human in the loop she will write an email to the server's administrator
              > to inform him of his bug in failing to implement the specification. It
              > is not the client's problem. It is the server's problem.
              >
              > > Atom (and any application protocol based on exchanging known document
              > > types) has to trade off between exhaustively specifying application
              > > behavior and exhaustively specifying failure handling.
              >
              > And exhaustively supporting forwards-compatability for extensions, and
              > exhaustively trading server and client-side complexity for protocol
              > features.
              >
              > > >If I come up to a server that has
              > > > support for an unexpectedly small subset of atom, I then have to
              > > > customize my content in an unexpected way.
              > > It is better for you to do the customizing. TTake the example of a
              > > server that simply cannot honor text/html or application/xhtml+xml in
              > > the title field. It can only handle text/plain. Atom protocol says
              > > nothing at the moment about this situation, except that the server can
              > > change your POSted data as it needs to. So presently my server either
              > > a) rejects your submission or b) stores it as text/plain. Better
              > > would be for your client to receive an Xform with a constraint
              > > specifiying "text/plain" only -- then, if the user had any imprtant
              > > rich content they wanted to put in the title, they can at least try to
              > > compensate.
              >
              > No no no.... the client doesn't know how to customise the content. As
              > the author of the client I relied on the atom specification that says I
              > can supply a content element and I did. Now your server is telling me it
              > doesn't understand it and wants a summary element instead? My client is
              > not written to deal with that. The server can deal with its own
              > shortcomings, thankyou. If it doesn't understand the protocol it should
              > stop speaking it.
              >
              > If the server doesn't understand xhtml in the title, then tough! It
              > knows that it must be expected to deal with xhtml in the title because
              > we agreed through the standardisation process that it should be capable.
              > How it provides that support as the atom spec rightly points out is up
              > to the server. Maybe it will strip out anything in angle-brackets before
              > storing the value into its internal title variable. Maybe it will just
              > use the xhtml content verbatim. That's the server's problem. It isn't
              > permitted to reject my submission. How is my client supposed to know
              > that what it needs to do in the face of this dumb server? We already
              > agreed that xhtml was fine, and now this server wants to go back on
              > that? Move complexity to me, will you? No thanks. I'll find another
              > server to talk to. Maybe one that understands xhtml in the title.
              >
              > > >That is to say, I way that
              > > > noone programmed me to customise my content in :)
              > > Well, you would have programmed it from this pov, so you would have
              > > handled these exceptional situations.
              >
              > I don't write client software to deal with broken servers that don't
              > implment the spec. It is up to the server to deal with the problem if it
              > can't translate my request precisely.
              >
              > > > The server, on the other hand, is in a good position to customise the
              > > > content. It knows which subset of atom it understands, and it knows what
              > > > atom is generally. It knows multiple authors might be required, so is in
              > > > a position to either model those multiple authors or use an algorithm to
              > > > select an author for its model from the available list.
              > > See above. Yes, you are describing the undesirable behavior the
              > > current APP forces you into.
              >
              > Quite the opposite. Your suggestion doesn't hold water. The spec
              > reflects decades of experience in developing protocols that work and can
              > evolve succesfully for decades to come. Do you think you can do better
              > without having written software on more than one side of the
              > client/server fence?
              >
              > > > > What I proposed is that the form delivered by a server have just the
              > > > > elements that make sense for the server. My server might not know what
              > > > > to do with a <source> element. Using the standard "form", my server
              > > > > has to have handling in place if you submit a <source> element, and it
              > > > > has to describe to you the problem if you do submit <source> and my
              > > > > server rejects it.
              > > > All atom elements make sense to the server, even if they don't fit the
              > > > server's internal model.
              > > > The server implements atom, after all.
              > > not so -- the client may submit extgension elements the server isn't aware of.
              >
              > And the server is required to ignore them and the client is required to
              > accept that old servers will ignore them. That's the way extensible
              > protocols work. If the extension is good it will be supported. If it is
              > not it will be ignored, sidelined, and eventually forgotten.
              >
              > > > What you
              > > > are suggesting is adding an extra message exchange to move the
              > > > complexity of fitting atom into the server's model back to the client.
              > > > Neither side is going to be great at solving a model mismatch problem,
              > > > but the client is likely to be downright incapable.
              > > As above: The client is in the best position to take corrective
              > > action, so as to best fulfill its intent.
              >
              > Show me the code. Your solution requires me to write client-side code
              > every time a new kind of dumb server is placed on the internet. That
              > doesn't work. That doesn't scale. Machines can't negotiate, only humans
              > can... and we already have. If you have anything more to add to that
              > conversation you had better to do it. Don't try to hold a separate
              > conversation with my client software. It doesn't know how to hold that
              > conversation.
              >
              > > Forms made the web adaptive. Go to any airline reservation web site.
              > > They're all the same, but different too. They mostly do the same
              > > things, but Orbitz offers packages with hotels and cars, while Delta
              > > offers connections with partner airlines, and Priceline won't let you
              > > see the itinerary. There's some shared vocabulary among all those
              > > sites but varying behavior. If the airlines standardize their
              > > vocabulary for forms, you could program a client to interact,
              > > adaptively, with all those sites. But you could not constrain
              > > Priceline's app to squeeze into the same behavior model as Delta's.
              > > And you shouldn't -- you should encourage diversity among web apps, as
              > > has been successful on the web to date.
              >
              > The evolvability of HTML and HTTP are what have been successful on the
              > web to date. Atom follows the evolvability and agreement model of its
              > predecessors. Do you really want your web server to have to retrieve a
              > form from every browser that requests a html page before it can be
              > returned? When that form says "I don't understand paragraph markers",
              > what will your web server do to make the content fit?
              >
              > Benjamin.
              >
              >
            • Bob Haugen
              ... That fits with my experience. Previous ecommerce standards, ANSI X12 EDI and EDIFACT, were even worse: took months. Some of the same people have now
              Message 6 of 25 , Jan 4, 2007
              • 0 Attachment
                On 1/3/07, Hugh Winkler <hughw@...> wrote:
                > Starry-eyed delusion: A Standards process can capture the union of all
                > the semantics any server could manifest.
                > Corollary: Exchanging these documents is "scalable"
                >
                > Real world experience: RosettaNet PIP 3A4. Thoroughly specified
                > schemas describing almost every possible Purchase Order Request and
                > Response. Guess how long it takes one large computer manufacturer to
                > shake out enough impedance mismatches to get going with each new
                > trading partner? 30-60 days, and that is a dedicated team of experts
                > exchanging test messages, "validating" them, discovering and resolving
                > the semantic mismatches in even the "valid" documents. So that's one
                > or two months per partner. How long does it take to ramp on just 100
                > partners? Years. Not web scale, is it?

                That fits with my experience. Previous ecommerce standards, ANSI X12
                EDI and EDIFACT, were even worse: took months.

                Some of the same people have now created UBL: a 4th or 5th generation
                (X12, EDIFACT, RosettaNet, UBL, depending on whether you put ebXML
                into that geneology). We'll see how quickly new trading partners can
                get going using that. I think it's less prescriptive than RosettaNet,
                which tried to achieve minimal configuration.
              • Benjamin Carlyle
                ... The actual query made by the server to the client is: Put a text box next to the human-readble text Please enter your email address . Return what the user
                Message 7 of 25 , Jan 4, 2007
                • 0 Attachment
                  On Wed, 2007-01-03 at 20:06 -0600, Hugh Winkler wrote:
                  > Objection: "Clients can't be programmed to complete forms"
                  > Counter-example: This machine automatically fills out a form based on
                  > a vocabulary every day: Your web browser, when it pre-fills form
                  > fields for you. Browsers know that user name and password fields on
                  > HTML forms usually have consistent names. And Google Toolbar saves
                  > some common personal info about you and auto-populates your name,
                  > address, etc, based on a common vocabulary. You just program a client
                  > to map information it has, to fields identified by name -- a
                  > vocabulary.
                  > Misconception: "Forms are schemas"
                  > Correction: Forms are queries made by the server to the client. "Give
                  > me the data you have named 'author-email". Schemas describe relations
                  > among objects. They describe models, not messages.

                  The actual query made by the server to the client is:
                  Put a text box next to the human-readble text "Please enter your email
                  address". Return what the user enters into that box as the
                  "author-email" element of an xml document. Oh, and by the way. Don't
                  submit it back until that xml document maches this xml schema I supply.

                  The kind of client atompub is designed to support already has a text
                  box. It's called "author". Now your form says "give me author-email"...
                  but there is no author-email box in the application. The application has
                  already been written and its data model doesn't match yours. Now what
                  would you like the client to do about it?

                  If you just want to supply a form to a user, you can do that. You don't
                  need atompub for that. You don't need any standardisation past HTML
                  forms or XForms. Atompub is what you are looking for when you want to
                  write a clever and easy to use thick-client atom authoring client.
                  Supplying a form to this thick client isn't going to help. You say you
                  want an author-email box next to the "Please enter your email address"
                  text? Hey! my interface has already been designed. Where do you expect
                  me to put that text box? If you want to design your own interface, just
                  supply the form to a web browser and be done with it. For me, I have my
                  own user interface. The purpose of atompub is to allow that client with
                  its complete user interface to submit articles to a server that has also
                  been written.

                  > Starry-eyed delusion: A Standards process can capture the union of all
                  > the semantics any server could manifest.
                  > Corollary: Exchanging these documents is "scalable"
                  >
                  > Real world experience: RosettaNet PIP 3A4. Thoroughly specified
                  > schemas describing almost every possible Purchase Order Request and
                  > Response. Guess how long it takes one large computer manufacturer to
                  > shake out enough impedance mismatches to get going with each new
                  > trading partner? 30-60 days, and that is a dedicated team of experts
                  > exchanging test messages, "validating" them, discovering and resolving
                  > the semantic mismatches in even the "valid" documents. So that's one
                  > or two months per partner. How long does it take to ramp on just 100
                  > partners? Years. Not web scale, is it?

                  Bad standards exist, therefore standards are bad. Nice logic. Next I'm
                  sure you'll tell me about how bad HTML is and why it will never be
                  deployed or understood by anyone. We should require that web browsers
                  provide a form for the server to fill out that informs the server of the
                  client's capabilities. That will help. When the client says it doesn't
                  understand <h3>, the server will know exactly what to do. It'll just
                  translate those h3 elements into h2 elements. Of course the programmer
                  who wrote the server will have anticipated this and written the
                  capability into it.

                  Of course the first round of standardisation is a shambles. It hardly
                  ever works until the authors of clients and servers have some
                  implementation experience and have developed some mutual trust in their
                  mutual interests. That is why standards like atom have defined
                  mechanisms for evolvability, and why atom has been so long in
                  development despite person-decades of experience in related precursor
                  standards.

                  Document standards represent a conversation between document producers
                  and consumers about their needs. It is a compromise between features and
                  complexity in problem domains that are rarely mapped out well enough to
                  have a sane conversation about. Trying to carry on a conversation
                  outside of the process however between your server and my client isn't
                  going to get anywhere. Machines can't communicate unless their
                  programmers already can. Unless the author of the client software and
                  the author of the server software agree out of band, communication
                  cannot happen.

                  > Exchanging pre-defined special purpose documents, as APP does,
                  > satisfies REST's definition, but it's not webby. Forms have proven
                  > their scalability on the web, because they promote loose coupling.
                  > They offer hope that web services could be deployed at web scale.

                  What you are saying is that server-provided user interfaces have proven
                  useful on the web because they don't require prior agreement with the
                  client software about what the user interface will look like. That is
                  all well and good, but doesn't translate into good practice for machine
                  to machine communications. When a machine has an atom document it wants
                  to submit to another machine, it is too late to start messing with the
                  content. The client machine doesn't know how. It has to rely on out of
                  band agreement with the server to what is acceptable.

                  Benjamin.
                • Bob Haugen
                  ... I don t think that s a valid conclusion. RosettaNet is not a bad standard. As I wrote in another post, it is the third generation of ecommerce standards
                  Message 8 of 25 , Jan 4, 2007
                  • 0 Attachment
                    On 1/4/07, Benjamin Carlyle <benjamincarlyle@...> wrote:
                    > On Wed, 2007-01-03 at 20:06 -0600, Hugh Winkler wrote:
                    > > Real world experience: RosettaNet PIP 3A4. Thoroughly specified
                    > > schemas describing almost every possible Purchase Order Request and
                    > > Response. Guess how long it takes one large computer manufacturer to
                    > > shake out enough impedance mismatches to get going with each new
                    > > trading partner? 30-60 days, and that is a dedicated team of experts
                    > > exchanging test messages, "validating" them, discovering and resolving
                    > > the semantic mismatches in even the "valid" documents.

                    > Bad standards exist, therefore standards are bad. Nice logic.

                    I don't think that's a valid conclusion. RosettaNet is not a bad
                    standard. As I wrote in another post, it is the third generation of
                    ecommerce standards that each tried to learn from and resolve the
                    problems of the previous generation. Each was better than its
                    predecessor. So by the Microsoft rule, RosettaNet should have been
                    pretty good.

                    It's just a difficult problem.
                  • Hugh Winkler
                    ... Exactly. RosettaNet is a great standard... the apotheosis of this style of exchanging full documents. My point is that style is not scalable, and that APP
                    Message 9 of 25 , Jan 4, 2007
                    • 0 Attachment
                      On 1/4/07, Bob Haugen <bob.haugen@...> wrote:
                      > On 1/4/07, Benjamin Carlyle <benjamincarlyle@...> wrote:
                      > > On Wed, 2007-01-03 at 20:06 -0600, Hugh Winkler wrote:
                      > > > Real world experience: RosettaNet PIP 3A4. Thoroughly specified
                      > > > schemas describing almost every possible Purchase Order Request and
                      > > > Response. Guess how long it takes one large computer manufacturer to
                      > > > shake out enough impedance mismatches to get going with each new
                      > > > trading partner? 30-60 days, and that is a dedicated team of experts
                      > > > exchanging test messages, "validating" them, discovering and resolving
                      > > > the semantic mismatches in even the "valid" documents.
                      >
                      > > Bad standards exist, therefore standards are bad. Nice logic.
                      >
                      > I don't think that's a valid conclusion. RosettaNet is not a bad
                      > standard. As I wrote in another post, it is the third generation of
                      > ecommerce standards that each tried to learn from and resolve the
                      > problems of the previous generation. Each was better than its
                      > predecessor. So by the Microsoft rule, RosettaNet should have been
                      > pretty good.
                      >
                      > It's just a difficult problem.
                      >
                      >

                      Exactly. RosettaNet is a great standard... the apotheosis of this
                      style of exchanging full documents. My point is that style is not
                      scalable, and that APP is using the same approach, on a smaller scale.
                    • Hugh Winkler
                      ... It would be good to experiment with a forms style using UBL. I think you could construct forms using UBL components, so that Toshiba could have their own
                      Message 10 of 25 , Jan 4, 2007
                      • 0 Attachment
                        On 1/4/07, Bob Haugen <bob.haugen@...> wrote:
                        > On 1/3/07, Hugh Winkler <hughw@...> wrote:
                        > > Starry-eyed delusion: A Standards process can capture the union of all
                        > > the semantics any server could manifest.
                        > > Corollary: Exchanging these documents is "scalable"
                        > >
                        > > Real world experience: RosettaNet PIP 3A4. Thoroughly specified
                        > > schemas describing almost every possible Purchase Order Request and
                        > > Response. Guess how long it takes one large computer manufacturer to
                        > > shake out enough impedance mismatches to get going with each new
                        > > trading partner? 30-60 days, and that is a dedicated team of experts
                        > > exchanging test messages, "validating" them, discovering and resolving
                        > > the semantic mismatches in even the "valid" documents. So that's one
                        > > or two months per partner. How long does it take to ramp on just 100
                        > > partners? Years. Not web scale, is it?
                        >
                        > That fits with my experience. Previous ecommerce standards, ANSI X12
                        > EDI and EDIFACT, were even worse: took months.
                        >
                        > Some of the same people have now created UBL: a 4th or 5th generation
                        > (X12, EDIFACT, RosettaNet, UBL, depending on whether you put ebXML
                        > into that geneology). We'll see how quickly new trading partners can
                        > get going using that. I think it's less prescriptive than RosettaNet,
                        > which tried to achieve minimal configuration.
                        >
                        >

                        It would be good to experiment with a forms style using UBL. I think
                        you could construct forms using UBL components, so that Toshiba could
                        have their own PO and Hitachi could have another. Purchasing software
                        would then be able to GET the forms, and complete them, POSTing the
                        right information in the PO, because the clients know the definitions
                        in UBL.

                        Hugh
                      • Mark Baker
                        ... http://sourceforge.net/projects/xforms4ubl/ Mark. -- Mark Baker. Ottawa, Ontario, CANADA. http://www.markbaker.ca Coactus; Web-inspired
                        Message 11 of 25 , Jan 4, 2007
                        • 0 Attachment
                          On 1/4/07, Hugh Winkler <hughw@...> wrote:
                          > It would be good to experiment with a forms style using UBL.

                          http://sourceforge.net/projects/xforms4ubl/

                          Mark.
                          --
                          Mark Baker. Ottawa, Ontario, CANADA. http://www.markbaker.ca
                          Coactus; Web-inspired integration strategies http://www.coactus.com
                        • Hugh Winkler
                          ... Excellent. So all we need to do is define UBL components for the parts of an atom:entry that don t already have definitions, e.g. ubl:atom-content... and
                          Message 12 of 25 , Jan 4, 2007
                          • 0 Attachment
                            On 1/4/07, Mark Baker <distobj@...> wrote:
                            > On 1/4/07, Hugh Winkler <hughw@...> wrote:
                            > > It would be good to experiment with a forms style using UBL.
                            >
                            > http://sourceforge.net/projects/xforms4ubl/
                            >

                            Excellent. So all we need to do is define UBL components for the parts
                            of an atom:entry that don't already have definitions, e.g.
                            ubl:atom-content... and you can publish to your blog using ubl-aware
                            clientware, plus you can order a hard disk too!

                            Hugh
                          • Benjamin Carlyle
                            ... I would suggest that by definition any standard that takes 30-60 days of experts exchanging test messages, validating them, discovering and resolving
                            Message 13 of 25 , Jan 5, 2007
                            • 0 Attachment
                              On Thu, 2007-01-04 at 06:37 -0600, Hugh Winkler wrote:
                              > On 1/4/07, Bob Haugen <bob.haugen@...> wrote:
                              > > On 1/4/07, Benjamin Carlyle <benjamincarlyle@...> wrote:
                              > > > On Wed, 2007-01-03 at 20:06 -0600, Hugh Winkler wrote:
                              > > > > Real world experience: RosettaNet PIP 3A4. Thoroughly specified
                              > > > > schemas describing almost every possible Purchase Order Request
                              > and
                              > > > > Response. Guess how long it takes one large computer
                              > manufacturer to
                              > > > > shake out enough impedance mismatches to get going with each new
                              > > > > trading partner? 30-60 days, and that is a dedicated team of
                              > experts
                              > > > > exchanging test messages, "validating" them, discovering and
                              > resolving
                              > > > > the semantic mismatches in even the "valid" documents.
                              > > > Bad standards exist, therefore standards are bad. Nice logic.
                              > > I don't think that's a valid conclusion. RosettaNet is not a bad
                              > > standard. As I wrote in another post, it is the third generation of
                              > > ecommerce standards that each tried to learn from and resolve the
                              > > problems of the previous generation. Each was better than its
                              > > predecessor. So by the Microsoft rule, RosettaNet should have been
                              > > pretty good.
                              > > It's just a difficult problem.
                              > Exactly. RosettaNet is a great standard... the apotheosis of this
                              > style of exchanging full documents. My point is that style is not
                              > scalable, and that APP is using the same approach, on a smaller scale.

                              I would suggest that by definition any standard that takes "30-60 days"
                              of "experts exchanging test messages, "validating" them, discovering and
                              resolving the semantic mismatches in even the "valid" documents" is a
                              bad standard. That doesn't mean the people are bad. It doesn't mean that
                              the ideas were bad. It just means that all the necessary ingredients of
                              standard prepration haven't come together in that problem domain as yet.

                              If it is working at all for anyone, I suppose it can't be all bad. Over
                              several more versions it may either become a good standard or form the
                              groundwork for a family of good standards. However it can't presently be
                              called good in the way that html is good. HTML works in a now familar
                              problem domain and can rely on common understanding and shared
                              motivation between participants about what features are necessary and
                              what they should look like. Standards like these can be scaled up with a
                              minimum of effort to a large number of users.

                              That is not to say that HTML was always a good standard. It needed to
                              evolve over time as more users placed more demands on it, and it did. It
                              also drew on earlier document standards and deployed hypertext systems
                              as a grounding for development.

                              Let me take your comments at their merit though. Hugh, you suggest that
                              the style of communication where people get together and agree on a
                              document format before communication takes place is not scalable. I'll
                              pay that in some respects. In the sense that we have clearly have good
                              standards today that scale up remarkably well, you are clearly mistaken.
                              However the process of standardisation is only as scalable as available
                              processes of human interaction and participation. That is to say it
                              tends to fracture on socio-political grounds.

                              Business is a hard problem-domain to crack because there are so many
                              people with different ideas about what business is. They each have their
                              own ways of working. They have their own management styles, styles of
                              dealing with clients and suppliers. They have their own vocabularies, so
                              it is hard coming up with a common vocabulary that means the same thing
                              to everyone. It's like coming up with a document format for common law
                              contracts. It can only be defined at a very high level with the real
                              semantics being hidden somewhere in human-readable text.

                              Atom is not in the business league. It is smaller and targetted to a set
                              of people who already know what a "blog" is, and can talk about it in
                              terms familiar to each other. It has still to pass the test of time that
                              HTML has and see off challenges such as hAtom, but I do tentatively
                              consider atom to be a good document standard.

                              Benjamin.
                            • Bill de hOra
                              ... I m not sure where to begin with this. I dislike muddled conversations when it comes to semantics. All the business processing standards I ve seen are
                              Message 14 of 25 , Jan 5, 2007
                              • 0 Attachment
                                Hugh Winkler wrote:
                                >
                                >
                                > On 1/4/07, Bob Haugen <bob.haugen@...
                                > <mailto:bob.haugen%40gmail.com>> wrote:
                                > > On 1/4/07, Benjamin Carlyle <benjamincarlyle@...
                                > <mailto:benjamincarlyle%40optusnet.com.au>> wrote:
                                > > > On Wed, 2007-01-03 at 20:06 -0600, Hugh Winkler wrote:
                                > > > > Real world experience: RosettaNet PIP 3A4. Thoroughly specified
                                > > > > schemas describing almost every possible Purchase Order Request and
                                > > > > Response. Guess how long it takes one large computer manufacturer to
                                > > > > shake out enough impedance mismatches to get going with each new
                                > > > > trading partner? 30-60 days, and that is a dedicated team of experts
                                > > > > exchanging test messages, "validating" them, discovering and
                                > resolving
                                > > > > the semantic mismatches in even the "valid" documents.
                                > >
                                > > > Bad standards exist, therefore standards are bad. Nice logic.
                                > >
                                > > I don't think that's a valid conclusion. RosettaNet is not a bad
                                > > standard. As I wrote in another post, it is the third generation of
                                > > ecommerce standards that each tried to learn from and resolve the
                                > > problems of the previous generation. Each was better than its
                                > > predecessor. So by the Microsoft rule, RosettaNet should have been
                                > > pretty good.
                                > >
                                > > It's just a difficult problem.
                                > >
                                > >
                                >
                                > Exactly. RosettaNet is a great standard... the apotheosis of this
                                > style of exchanging full documents. My point is that style is not
                                > scalable, and that APP is using the same approach, on a smaller scale.

                                I'm not sure where to begin with this. I dislike muddled conversations
                                when it comes to semantics.

                                All the business processing standards I've seen are inherently
                                non-scalable, if by non-scalable you mean the number of actors that can
                                share information without resorting to effort in out of band agreements.
                                That XForms has a means of exception management (show it to a person)
                                and does schema validation in advance via constraints instead of
                                defensive programming doesn't have me thinking it's any more applicable
                                or valuable than using Atom - indeed it has me thinking it's going to be
                                riddled with security holes because to honor an xform constraint you
                                need a trustworthy client.

                                In the general case this a symbolic AI problem. If you want to move
                                further along, the applicable state of the art here are denotational
                                semantics as used in KR, RDF interchange, or agent based systems like
                                FIPA. No-one seems to be sure how to design such things in a way that
                                they garner wide adoption on existing infrastructure.

                                cheers
                                Bill
                              • Hugh Winkler
                                ... Yes, that s what I mean. ... I am wide open to suggestions that Xforms may not be up to the task. I suggested Xfroms because there is a spec for it, and
                                Message 15 of 25 , Jan 5, 2007
                                • 0 Attachment
                                  On 1/5/07, Bill de hOra <bill@...> wrote:
                                  > Hugh Winkler wrote:

                                  > > Exactly. RosettaNet is a great standard... the apotheosis of this
                                  > > style of exchanging full documents. My point is that style is not
                                  > > scalable, and that APP is using the same approach, on a smaller scale.
                                  >
                                  > All the business processing standards I've seen are inherently
                                  > non-scalable, if by non-scalable you mean the number of actors that can
                                  > share information without resorting to effort in out of band agreements.

                                  Yes, that's what I mean.

                                  > That XForms has a means of exception management (show it to a person)
                                  > and does schema validation in advance via constraints instead of
                                  > defensive programming doesn't have me thinking it's any more applicable
                                  > or valuable than using Atom - indeed it has me thinking it's going to be
                                  > riddled with security holes because to honor an xform constraint you
                                  > need a trustworthy client.

                                  I am wide open to suggestions that Xforms may not be up to the task. I
                                  suggested Xfroms because there is a spec for it, and since it's all
                                  about XML, you can make a small leap and declare that the Relax NG and
                                  the RFC define the meanings of elements. So you can sidestep opening
                                  the semantic web discussion.

                                  Constraints just emulate what an HTML form indicates to humans: The
                                  HTML form puts one line for authors, not 10, so the browser cannot
                                  possibly accept more than a single author to submit to the server.

                                  [Side note about exceptions: even in a big machine to machine system,
                                  exceptional conditions usually have to percolate their way back to a
                                  human for handling. It may just be an error log, and the solution may
                                  be... Dang, our purchasing agent/blog syndicator/whatever needs to
                                  understand this new term the server at XYZ.com is demanding. ]

                                  >
                                  > In the general case this a symbolic AI problem. If you want to move
                                  > further along, the applicable state of the art here are denotational
                                  > semantics as used in KR, RDF interchange, or agent based systems like
                                  > FIPA. No-one seems to be sure how to design such things in a way that
                                  > they garner wide adoption on existing infrastructure.
                                  >

                                  Yeah, well I was hoping this discussion would begin the thinking about
                                  "wide adoption on existing infrastructure". Because if Xforms isn't
                                  up to it, or if RDF forms cannot be brought along far enough, to
                                  handle a simple little web service like APP, then we'll never see web
                                  services adopted as broadly as the web -- which the purpose of a lot
                                  of discussion on this list.

                                  My thinking is that APP is bite-sized enough to where some small
                                  unambitious forms language could lead the way... "microforms". I don't
                                  think it's going to take an Apollo project and a bunch of egghead
                                  professors to start this work... just another little brushfire of
                                  web innovation.

                                  Hugh
                                • Benjamin Carlyle
                                  ... But again, it isn t a matter of the protocol being up to it or not. It is about whether the client is up to it. The thing is that you can t communicate
                                  Message 16 of 25 , Jan 6, 2007
                                  • 0 Attachment
                                    On Fri, 2007-01-05 at 14:41 -0600, Hugh Winkler wrote:
                                    > On 1/5/07, Bill de hOra <bill@...> wrote:
                                    > > Hugh Winkler wrote:
                                    > > > Exactly. RosettaNet is a great standard... the apotheosis of this
                                    > > > style of exchanging full documents. My point is that style is not
                                    > > > scalable, and that APP is using the same approach, on a smaller
                                    > scale.
                                    > > All the business processing standards I've seen are inherently
                                    > > non-scalable, if by non-scalable you mean the number of actors that
                                    > can
                                    > > share information without resorting to effort in out of band
                                    > agreements.
                                    > Yes, that's what I mean.
                                    ...
                                    > Yeah, well I was hoping this discussion would begin the thinking about
                                    > "wide adoption on existing infrastructure". Because if Xforms isn't
                                    > up to it, or if RDF forms cannot be brought along far enough, to
                                    > handle a simple little web service like APP, then we'll never see web
                                    > services adopted as broadly as the web -- which the purpose of a lot
                                    > of discussion on this list.

                                    But again, it isn't a matter of the protocol being up to it or not. It
                                    is about whether the client is up to it. The thing is that you can't
                                    communicate without human-level agreement. This applies to protocols
                                    built around document standards in which the agreement is encoded
                                    directly into client and server. It also applies to protocols built
                                    around forms. In the forms case the server provides enough information
                                    in the form such that a human-level intelligence can figure out what is
                                    meant by all of the fields and attempt to fill them in correctly. The
                                    form is a user inteface, and the same kind of human intelligence needs
                                    to be applied to any filling in or manipulating any user interface.

                                    The terminolgy has to meet basic criteria of collective understanding
                                    even with human-level intelligence. The human must know what an author
                                    is in order to fill out the author field, and map it first to the
                                    concept of "well, that's me" and then to their own name as they would
                                    like it to appear. They might have to take a guess at how the
                                    information will be used to know what form of their name you want. They
                                    may have to refer to a help page to understand all of the necessary
                                    intricacies involved.

                                    So I think you really want to to ask are the dual queries of "How to we
                                    prevent technology issues getting in the way of human-level agreement?"
                                    and "How do we best achieve human-level agreement in a world of
                                    subcultures and individuals?".

                                    The machine document-exchange style relies on agreement being made
                                    before the communication. The forms-exchange style relies on agreement
                                    being made with a human in the loop during the conversation as they
                                    determine how to fill out the form in the way the form provider
                                    intended. An incorrectly filled-out form is a failure to agree. A
                                    correctly filled-out form is the result of agreement via the form
                                    monalogue being understood sufficiently well by the form user.

                                    I think there is a sliding scale to this. Will all atom publication use
                                    atompub? Of course not. Most servers will allow users to submit new
                                    articles online using their web browser. However, when there is no human
                                    in the loop at the time of submission or it is difficult to engage the
                                    user in the communications process you need prior agreement. Consider
                                    the program that allowed a user to write their article offline. Now the
                                    client has atom-equivalent data to submit. It doesn't want to fill out a
                                    form, and can do very little to have the user submit different content
                                    if the server fails to accept the submission.

                                    You need good agreement where you want documents to be understood for
                                    decades as well. After all, when human agreement is required to be
                                    sustained for a long time it also takes a long time to come to a point
                                    of agreement.

                                    Within the machine document-exchange style there is also a sliding
                                    scale. I could agree between my client software and your server software
                                    on a document format. This wouldn't require standardisation over a large
                                    user base, although we are still defining a standard. This would be our
                                    own vocabulary not understood by anyone else. Essentially, our network
                                    effects would be limited to the size of the network between you and me.

                                    On the other end of the scale are your htmls, atoms and other
                                    widely-agreed document formats. These require a lot more effort to
                                    agree, and more effort to change once the initial agreement has come
                                    about. They will be understood for a long time by a large number of
                                    people. They won't be costly to roll out into new applications. But then
                                    again, they are limited in the set of problems they can solve. They can
                                    only solve problems that affect a majority or significant minority of
                                    the standard's contributors.

                                    If we constrast this scale to the development of human language we can
                                    see that humans generally know a small number of general purpose
                                    languages that can solve common problems. Uncommon problems tend to
                                    require specialised vocaulary. Various sciences and technology problem
                                    domains will often have specific jargon that allows uncommon problems to
                                    be solved. I think that the way we define documents for machines will
                                    eventually follow this kind of trend. The jargon could be defined as
                                    part of problem-specific document standards or as adjuncts to more
                                    general standards.

                                    Perhaps our common language for expressing client/server interaction is
                                    HTML, but within html we might define a special language (hCalendar) for
                                    expressing information in the time and date problem domain. Perhaps
                                    revealingly, this kind of mixed vocabulary set can also be combined with
                                    form submission for very special purpose problem domains that might
                                    apply only to a single server on the Internet. These problem domains
                                    afford the least automation and require the greatest application of
                                    human-level intelligence... but a speical problem domain is a special
                                    problem domain.

                                    I think an important focus for evolving the general concept of document
                                    definition needs to be built around allowing not only evolution of the
                                    global concepts, but inclusion of vocabularies that may be
                                    problem-specific. The main reason that this is difficult is because over
                                    time our perspective on the vocabularies may change. What was once
                                    standard might one day be shelved as a special vocabulary. What was once
                                    specal may become part of a standard vocabulary. This process can be
                                    difficult when various special vocabularies overlap or conflict in any
                                    way.

                                    I think forms are important. Don't get me wrong. However, I think that
                                    the basic approach of using documents to pass information between
                                    machines will be an important one for the forseeable future. These
                                    documents must solve most problems with a general vocabulary, but may
                                    still require special vocabularies to be built on top of the general and
                                    combined in interesting ways. If this makes me sound like I am on the
                                    RDF bandwagon, well.. I'm yet to be sold.

                                    I'm not convinced RDF solves the evolution problem as particular terms
                                    change in meaning or empahsis over time. I'm pretty sure I don't want my
                                    documents to have to define ten namespaces at the top and make sure I
                                    select terms from the appropriate namespace for every element in the
                                    file. Perhaps the problem is as simple as placing a <problem-domain>
                                    marker into atom for any special problem domain, and allowing its
                                    sub-elements to float in and out of the main atom feed and entry
                                    elements over subsequent atom versions. Perhaps a special <vendor>
                                    elment would make sense to collect anything specific to a particular
                                    application. The important thing would be to allow these special
                                    vocabularies to evolve separately to atom proper as required.

                                    Benjamin.
                                  Your message has been successfully submitted and would be delivered to recipients shortly.