Loading ...
Sorry, an error occurred while loading the content.

Re: [json] JSON.hpack - JavaScript, PHP, C# (Python Soon)

Expand Messages
  • Tatu Saloranta
    On Wed, May 27, 2009 at 6:27 AM, Andrea Giammarchi ... This seems like a simple and possibly sensible convention. But in addition to having an implementation,
    Message 1 of 8 , May 27 12:08 PM
    • 0 Attachment
      On Wed, May 27, 2009 at 6:27 AM, Andrea Giammarchi
      <andrea.giammarchi@...> wrote:
      > (it was a direct message, I have copied and pasted here as well)
      > Good afternoon Mr Douglas Crockford,
      > I wonder if you have seen already my latest JSON related project, called
      > JSON.hpack,
      > an Homogeneous Collection Packer.
      >
      > As summary, hpack transform a JSON list of objects, usually a database
      > resultSet
      > [{name:value},{name:value},{name:value}]
      > into an array only collection with columns info in the special index 0

      This seems like a simple and possibly sensible convention.
      But in addition to having an implementation, it would seem most
      interesting to specify the convention. This would allow
      implementations for other languages: verbosity of textual data formats
      matters for many use cases, and json is being used a lot on
      server-to-server communication too.

      Wrt compression: I am sure gzip would yield more additional
      compression, since it would not only eliminate redundancy wrt. names
      (which compress very well obviously) but also for values. It can
      definitely be added as a separate layer.
      But given how CPU hungry gzip is (see
      [http://www.cowtowncoder.com/blog/archives/2009/05/entry_263.html%5d for
      example; for java json, overhead is 3x - 5x basic processing),
      conventions can help a lot, so the two can be complementary; even if
      plain gzip would probably compress things more if size reduction was
      the only goal.

      -+ Tatu +-
    • an_red@ymail.com
      Tatu, gzip works faster in any case over JSON.hpack(ed) stuff for the same reason JSON.stringify / parse is faster over JSON.hpacked stuff, less characters to
      Message 2 of 8 , May 28 1:25 AM
      • 0 Attachment
        Tatu, gzip works faster in any case over JSON.hpack(ed) stuff for the same reason JSON.stringify / parse is faster over JSON.hpacked stuff, less characters to work over (you gzip a string smaller up to 80%)

        It could be a separated layer in any case, but even over, will be faster. The C# implementation of both hpack and hunpack are extremely fast, I need time to create a Python version and I was planning to create a PHP extension in C as well but only one Java guy told me he will try to create a parser ... I mean, C# version is not that different, I guess a porting will take a couple of hours and nothing else.

        Anyway, I miss your point about specify the convention ... do you mean proper specs? I could write something more than I already wrote on github homepage, let me know.

        Regards

        --- In json@yahoogroups.com, Tatu Saloranta <tsaloranta@...> wrote:
        >
        > On Wed, May 27, 2009 at 6:27 AM, Andrea Giammarchi
        > <andrea.giammarchi@...> wrote:
        > > (it was a direct message, I have copied and pasted here as well)
        > > Good afternoon Mr Douglas Crockford,
        > > I wonder if you have seen already my latest JSON related project, called
        > > JSON.hpack,
        > > an Homogeneous Collection Packer.
        > >
        > > As summary, hpack transform a JSON list of objects, usually a database
        > > resultSet
        > > [{name:value},{name:value},{name:value}]
        > > into an array only collection with columns info in the special index 0
        >
        > This seems like a simple and possibly sensible convention.
        > But in addition to having an implementation, it would seem most
        > interesting to specify the convention. This would allow
        > implementations for other languages: verbosity of textual data formats
        > matters for many use cases, and json is being used a lot on
        > server-to-server communication too.
        >
        > Wrt compression: I am sure gzip would yield more additional
        > compression, since it would not only eliminate redundancy wrt. names
        > (which compress very well obviously) but also for values. It can
        > definitely be added as a separate layer.
        > But given how CPU hungry gzip is (see
        > [http://www.cowtowncoder.com/blog/archives/2009/05/entry_263.html%5d for
        > example; for java json, overhead is 3x - 5x basic processing),
        > conventions can help a lot, so the two can be complementary; even if
        > plain gzip would probably compress things more if size reduction was
        > the only goal.
        >
        > -+ Tatu +-
        >
      • Tatu Saloranta
        On Thu, May 28, 2009 at 1:25 AM, an_red@ymail.com ... Yes, certainly, not arguing with that, wrt. speed. ... No doubt, it should be a fairly simple
        Message 3 of 8 , May 28 4:39 PM
        • 0 Attachment
          On Thu, May 28, 2009 at 1:25 AM, an_red@...
          <andrea.giammarchi@...> wrote:
          > Tatu, gzip works faster in any case over JSON.hpack(ed) stuff for the same reason JSON.stringify / parse is faster over JSON.hpacked stuff, less characters to work over (you gzip a string smaller up to 80%)

          Yes, certainly, not arguing with that, wrt. speed.

          > It could be a separated layer in any case, but even over, will be faster. The C# implementation of both hpack and hunpack are extremely fast, I need time to create a Python version and I was planning to create a PHP

          No doubt, it should be a fairly simple transformation on other
          languages as well.

          > extension in C as well but only one Java guy told me he will try to create a parser ... I mean, C# version is not that different, I guess a porting will take a

          There are many good fast Java parsers, so I would recommend using one
          of those, and add transformation layer on top. No need to write a
          parser (IMO -- but I have written mine so I may be biased).

          > Anyway, I miss your point about specify the convention ... do you mean
          > proper specs? I could write something more than I already wrote on github homepage, let me know.

          Yeah, specification of this convention (or transformation),
          documenting how it should behave. This is necessary (or at least
          useful) to ensure interoperability. Given how many people want to
          write their own json parsers (... just a random example...), because
          they don't like what is available, I'm sure there'll be many who'd
          want to reimplement this mapping as well.
          But at least implementations would be interoperable.

          So yes, need not be anything too heavy-weight, but somewhat formal
          notation helps in my opinion.

          Thanks!

          -+ Tatu +-
        • an_red@ymail.com
          Is something like this enough? http://wiki.github.com/WebReflection/json.hpack/specs-details Best Regards. Andrea Giammarchi
          Message 4 of 8 , May 29 3:29 AM
          • 0 Attachment
            Is something like this enough?
            http://wiki.github.com/WebReflection/json.hpack/specs-details

            Best Regards.

            Andrea Giammarchi

            --- In json@yahoogroups.com, Tatu Saloranta <tsaloranta@...> wrote:
            >
            > On Thu, May 28, 2009 at 1:25 AM, an_red@...
            > <andrea.giammarchi@...> wrote:
            > > Tatu, gzip works faster in any case over JSON.hpack(ed) stuff for the same reason JSON.stringify / parse is faster over JSON.hpacked stuff, less characters to work over (you gzip a string smaller up to 80%)
            >
            > Yes, certainly, not arguing with that, wrt. speed.
            >
            > > It could be a separated layer in any case, but even over, will be faster. The C# implementation of both hpack and hunpack are extremely fast, I need time to create a Python version and I was planning to create a PHP
            >
            > No doubt, it should be a fairly simple transformation on other
            > languages as well.
            >
            > > extension in C as well but only one Java guy told me he will try to create a parser ... I mean, C# version is not that different, I guess a porting will take a
            >
            > There are many good fast Java parsers, so I would recommend using one
            > of those, and add transformation layer on top. No need to write a
            > parser (IMO -- but I have written mine so I may be biased).
            >
            > > Anyway, I miss your point about specify the convention ... do you mean
            > > proper specs? I could write something more than I already wrote on github homepage, let me know.
            >
            > Yeah, specification of this convention (or transformation),
            > documenting how it should behave. This is necessary (or at least
            > useful) to ensure interoperability. Given how many people want to
            > write their own json parsers (... just a random example...), because
            > they don't like what is available, I'm sure there'll be many who'd
            > want to reimplement this mapping as well.
            > But at least implementations would be interoperable.
            >
            > So yes, need not be anything too heavy-weight, but somewhat formal
            > notation helps in my opinion.
            >
            > Thanks!
            >
            > -+ Tatu +-
            >
          • Tatu Saloranta
            On Fri, May 29, 2009 at 3:29 AM, an_red@ymail.com ... I think so (didn t read in detail) -- and if there are questions, it s easy to contact you and suggest
            Message 5 of 8 , May 29 10:22 AM
            • 0 Attachment
              On Fri, May 29, 2009 at 3:29 AM, an_red@...
              <andrea.giammarchi@...> wrote:
              > Is something like this enough?
              > http://wiki.github.com/WebReflection/json.hpack/specs-details

              I think so (didn't read in detail) -- and if there are questions, it's
              easy to contact you and suggest clarifications.

              Thanks!

              -+ Tatu +-
            Your message has been successfully submitted and would be delivered to recipients shortly.