On Wed, Sep 21, 2011 at 10:44 AM, Stephan Beal <sgbeal@...
> On Wed, Sep 21, 2011 at 6:45 PM, Tatu Saloranta <tsaloranta@...>wrote:
>> You might be interested in an existing such specification called
>> Smile: http://wiki.fasterxml.com/SmileFormatSpec
>> which was specified about a year ago, has Java and C implementations,
>> and used by a few projects/products like ElasticSearch.
> Thank you for that. Smile's requirement that impls be capable of supporting
> "shared strings" seems a bit draconian to me, though. That adds non-trivial
> parser/writer infrastructure which would otherwise not be required
> (especially in C, which doesn't have standard containers we can use to store
> such strings/references in).
Correct, format does not aim for minimal complexity of implementations.
But for space efficiency it is pretty much a requirement as small set
of names is typically reused over and over again for data streams, and
back reference use can reduce size significantly.
This is an optional feature for encoder for what it is worth.
-+ Tatu +-