Loading ...
Sorry, an error occurred while loading the content.

Re: [json] standard content-type for "streaming JSON" (newline delimited JSON objects)?

Expand Messages
  • Stephan Beal
    ... Wouldn t a standards-compliant (or sane?) parser stop reading at the end of the first object (implying that the first above is, strictly speaking, standard
    Message 1 of 3 , May 23, 2012
    • 0 Attachment
      On Wed, May 23, 2012 at 11:10 PM, Ken Woodruff <ken.woodruff@...>wrote:

      > **
      >
      > { "id": 1, "foo": "bar" }
      > { "id": 2, "foo": "baz" }
      > ...
      >
      > This format seems both pragmatic and useful, but is not JSON compliant
      > (i.e. doesn't parse with a standard JSON parser)
      >

      Wouldn't a standards-compliant (or sane?) parser stop reading at the end of
      the first object (implying that the first above is, strictly speaking,
      standard JSON with extra crud at the end)?

      If so... adding the convention "if the byte following the root node is a
      newline or \r\n combination, attempt to parse remaining bytes as JSON..."
      might be more or less all you need. Such an extension would need to address
      what to do (or not do) with the consumed 1-2 bytes if they are _not_ \n or
      \r\n, but those are details we'll leave to the spec writers...

      Happy Hacking!

      --
      ----- stephan beal
      http://wanderinghorse.net/home/stephan/
      http://gplus.to/sgbeal


      [Non-text portions of this message have been removed]
    • Matthew Morley
      ... This is came up a while back, we were discussing streams of json over sockets/tcp, in relation to json-rpc streams here:
      Message 2 of 3 , May 23, 2012
      • 0 Attachment
        On Wed, May 23, 2012 at 5:10 PM, Ken Woodruff <ken.woodruff@...>wrote:

        > **
        >
        >
        > There seem to be a growing number of tools & packages around that
        > implement some form of JSON streaming where multiple standard JSON
        > objects are delimited by extra newlines, e.g.
        >
        > { "id": 1, "foo": "bar" }
        > { "id": 2, "foo": "baz" }
        > ...
        >
        > This format seems both pragmatic and useful, but is not JSON compliant
        > (i.e. doesn't parse with a standard JSON parser), so it seems
        > inappropriate to serve up as "application/json". The NPM package
        > Request-JSONStream uses "application/jsonstream", Google searching
        > shows at least one use of "application/x-json-stream", and there are a
        > number of services that use "application/json" and expect clients to
        > just deal with it (cf.
        > https://github.com/senchalabs/connect/issues/538).
        >
        > Since I'm about to make heavy use of this technique in a way that will
        > be a little difficult to unwind later I'd like if at all possible get
        > on board with whatever will become the standard (defacto or official).
        > Anyone aware of any efforts underway to standardize this, or
        > packages/services that have enough momentum to drive a standard in the
        > future?
        >
        > --Ken
        >
        >

        This is came up a while back, we were discussing streams of json over
        sockets/tcp, in relation to json-rpc streams here:

        https://groups.google.com/forum/?fromgroups#!msg/json-rpc/fd1_anYEnDs/5k_w5Vu9XtMJ

        Use of the ascii character 0x04 EOT was also an option others have
        suggested in these discussions.

        Using newlines seem like the same topic, just a different end of json
        marker. There are a lot of ways to achieve this of course, EOL markers seem
        as straight forward of a solution as any. The ability for a single bad json
        object in the stream to corrupt parsing down the line always seemed like a
        concern, perhaps unavoidable and unlikely at the same time?

        I still feel like sending a json array with a parser capable of spinning
        out parsed array values as soon as it can makes the most sense, like
        SAX<http://en.wikipedia.org/wiki/Simple_API_for_XML> based
        parsing. But I'll be glad to see a standard/best practice come of out this
        thread.

        --
        Matt (MPCM)


        [Non-text portions of this message have been removed]
      Your message has been successfully submitted and would be delivered to recipients shortly.