1249yajl-ruby: Ruby C bindings to Yajl - a sax-like streaming JSON parser
- May 5, 2009Hey everyone,
Just wanted to announce my new ruby C bindings to Yajl creatively called: yajl-ruby ;)
The sources are up on Github for now at http://github.com/brianmario/yajl-ruby although I might put it up on RubyForge soon too. Instructions on how to install and use are in the README.rdoc file.
To my knowledge this is only the second C-based JSON parser for Ruby aside from the JSON gem that's existed for a while.
In my preliminary parsing benchmarks, yajl-ruby performed on-par (in terms of time to completion) with the JSON gem on small-medium sized JSON strings (385 bytes to around 25k) but performs significantly better on larger files (100k+).
But as far as memory usage is concerned - it's much more efficient. For example: parsing a 2.4MB JSON string, ruby was using around 30MB of RAM for yajl-ruby and almost 80MB for the JSON gem. Using the OSX Instruments development tool, doing the same operation was showing 13MB compared to 42MB of allocations respectively.
Parsing as a stream opens up a whole new realm of possibilities for using JSON to transfer large amounts of data. I have some examples, including Yajl::HttpStream.get(uri) which will parse the JSON response body *directly off the socket* for the highest possible efficiency. Or even directly from a compressed Gzip or Bzip2 stream without the need to decompress to a buffer, then parse. I have tons of feature ideas around Http and Socket communication in general. But I'd love to hear yours!
My next big API addition will be streaming JSON string creation using Yajl. Then I'll work on some examples of using yajl-ruby on both sides of a stream.
Please check it out and give me any/all improvement suggestions and/or feature requests you might have.