Re: [json] Re: Universal Binary JSON Specification
- Yes, that is what I was getting at. But see comments embedded.
On Wed, Sep 21, 2011 at 7:50 PM, rkalla123 <rkalla@...> wrote:
> I see your point. The way I understand it is that this would require 2 new
> data types, effectively BigInt and BigDecimal.
> So say something along these lines:
> bigint - marker 'G'
> [G][129 big-endian ordered bytes representing a BigInt]
> It should be mentioned that they are signed ints, but doing two's
complement and such is probably too much work. Maybe just specify that the
first bit always represents the sign (0 for no sign, 1 or minus).
> bigdouble - marker 'W'
> [W][222 big-endian ordered bytes representing a BigDecimal]
BigDecimal should probably be renamed to something like BigFloat, since
decimal is ambiguous (used to mean base-10 and floating point). I'm less
familiar with large floating point, but I think a floating point number
should consist of a sign bit plus two integers (one for the
mantissa/significand and one for the exponent). In the interest of space
savings, I think the sign bit should just be included in the exponent and
order things so they look similar to the IEEE 754 spec, e.g.,
[W][3 big-endian ordered bytes (where first bit is sign bit) of
exponent][222 big-endian ordered bytes of mantissa]
In terms of the documentation, I think the big integers and floats should be
qualified with a "should implement" instead of a "must implement", since, as
others have mentioned, not every encoder and decoder will be able to handle
these. I think this matches JSON implementations well. If an encoder does
not handle large numbers, it could just throw an error, just as it should
throw an error now if an oversized number is encountered in JSON. The same
goes for the decoder side. If there is no good way to represent a large
number in the language your are working in, throw an error indicating that
the number is too large.
Have you looked into using variable-length integers for length specifiers?
If you have a lot of short strings (or big numbers, etc.) in your data,
these could significantly reduce your space usage (at the cost of more
complexity for the developer and CPU). There should be a balance between
space efficiency and complexity. Thoughts?
> --- In email@example.com, Don Owens <don@...> wrote:
> > I've seen very large numbers used in JSON. In Perl, that can be
> > as a Math::BigInt object. And that is the way I have implemented it in my
> > JSON module for Perl (JSON::DWIW). Python has arbitrary length integers
> > built-in. For my own language that I'm working on, I'm using libgmp in C
> > handle arbitrary length integers.
> > JSON is used as a data exchange format. I want to be able to do a
> > roundtrip, e.g., Python -> encoded -> Python with native integers (with
> > arbitrary length in this case). In JSON, this just works, as far as the
> > encoding is concerned. I see the need for this in any binary JSON format
> > well. If a large number is represented as a string, then on the decoding
> > side, you don't know if that was a number or a string (just because it
> > like a number doesn't mean that the sender means it's a number). If, when
> > decoding JSON, the library can't handle large numbers, it has to throw an
> > error anyway. The same should go for binary JSON.
> > ./don
[Non-text portions of this message have been removed]
- On Mon, Feb 20, 2012 at 9:42 AM, rkalla123 <rkalla@...> wrote:
> Stephan,For what it is worth, I also consider support for only signed values a
> No problem; your feedback are still very applicable and much appreciated.
> The additional view-point on the signed/unsigned issue was exactly what I was hoping for. My primary goal has always been simplicity and I know at least from the Java world, going with unsigned values would have made the impl distinctly *not* simple (and an annoying API).
> So I am glad to get some validation there that I am not alienating every other language at the cost of Java.
-+ Tatu +-