Loading ...
Sorry, an error occurred while loading the content.

Re: A new java JSON API + Up to date benchmarks and test for most existing Json API.

Expand Messages
  • uriel.chemouni
    Each test has a tag, to show if the test is a standard test, or not. I have to add this information in my wiki page. The score line is incremented by 100 by
    Message 1 of 18 , May 16 4:43 AM
    • 0 Attachment
      Each test has a tag, to show if the test is a standard test, or not.

      I have to add this information in my wiki page.

      The score line is incremented by 100 by standard test, 10 by non-strict test (like using ' instead of "), and 1 for strange test.

      --- In json@yahoogroups.com, Petri Lehtinen <petri@...> wrote:
      >
      > uriel.chemouni wrote:
      > > Hi
      > >
      > > I have recently published my Json API.
      > >
      > > Features test:
      > > http://code.google.com/p/json-smart/wiki/FeaturesTests
      > >
      > > Benches test:
      > > http://code.google.com/p/json-smart/wiki/Benchmark
      > >
      > > Project home;
      > > http://code.google.com/p/json-smart
      > >
      > > Don't hesitate to send me your feedback.
      >
      > It seems you have implemented your own data format that you call
      > "smaller JSON data". It's wrong to call this format JSON as it
      > violates the JSON spec in many ways.
      >
      > It's also very misleading and unfair to use this syntax when comparing
      > to other JSON libraries, as if they were supposed to support this
      > non-JSON data format. More than half of the test cases you list on the
      > FeatureTests page are not JSON at all, but still you expect others to
      > parse them without throwing an error.
      >
    • Uriel Chemouni
      I have change features test page, and separate standard and non-standard Tests. thanks for this comments.. ... [Non-text portions of this message have been
      Message 2 of 18 , May 16 6:26 AM
      • 0 Attachment
        I have change features test page, and separate standard and non-standard
        Tests.

        thanks for this comments..


        On Mon, May 16, 2011 at 12:46 PM, Petri Lehtinen <petri@...> wrote:

        > uriel.chemouni wrote:
        > > Hi
        > >
        > > I have recently published my Json API.
        > >
        > > Features test:
        > > http://code.google.com/p/json-smart/wiki/FeaturesTests
        > >
        > > Benches test:
        > > http://code.google.com/p/json-smart/wiki/Benchmark
        > >
        > > Project home;
        > > http://code.google.com/p/json-smart
        > >
        > > Don't hesitate to send me your feedback.
        >
        > It seems you have implemented your own data format that you call
        > "smaller JSON data". It's wrong to call this format JSON as it
        > violates the JSON spec in many ways.
        >
        > It's also very misleading and unfair to use this syntax when comparing
        > to other JSON libraries, as if they were supposed to support this
        > non-JSON data format. More than half of the test cases you list on the
        > FeatureTests page are not JSON at all, but still you expect others to
        > parse them without throwing an error.
        >


        [Non-text portions of this message have been removed]
      • Tatu Saloranta
        On Mon, May 16, 2011 at 3:16 AM, uriel.chemouni ... Performance test section could surely use some work. It tells very little, but makes big unsubstantiated
        Message 3 of 18 , May 16 8:53 AM
        • 0 Attachment
          On Mon, May 16, 2011 at 3:16 AM, uriel.chemouni
          <uriel.chemouni@...> wrote:
          > Hi
          >
          > I have recently published my Json API.
          >
          > Features test:
          > http://code.google.com/p/json-smart/wiki/FeaturesTests
          >
          > Benches test:
          > http://code.google.com/p/json-smart/wiki/Benchmark
          >
          > Project home;
          > http://code.google.com/p/json-smart
          >
          > Don't hesitate to send me your feedback.

          Performance test section could surely use some work. It tells very
          little, but makes big unsubstantiated claims regarding performance.

          For example:

          - where is source?
          - JSON used?
          - why is JVM not warmed up -- by numbers it looks like numbers are
          dominated by JVM/HotSpot runup times
          - why not use one of existing frameworks to get at least some
          semblance of due diligence wrt run times, variances and so on?

          I also agree in that it is wrong to call this json anything, given
          that it is for data format that looks a bit like JSON; should be
          called something else.
          And from this, comparing performance to real JSON parsers is somewhat
          odd, since parsers actually work on different data formats.

          -+ Tatu +-
        • uriel.chemouni
          Hi Tatu, ... I can publish (I m not shure) a working copy of the benching software, due to dépendances licence. However I have just upload the full benchmark
          Message 4 of 18 , May 16 9:41 AM
          • 0 Attachment
            Hi Tatu,
            > - where is source?

            I can publish (I'm not shure) a working copy of the benching software, due to dépendances licence.
            However I have just upload the full benchmark package in the SVN of json-smart:
            http://json-smart.googlecode.com/svn/trunk/bench/

            > - JSON used?
            json data look ,like:
            {"K1":"V1","K2":"V2","K3":"V3"}
            Key can be: "firstname", "lastname", "date", "len", "shape", "gate", "foo", "bar", "city", "site", "url", "age", "action", "level", "password", "color", "case"

            V* : depending of the test (int, float, unicode, boolean...)

            see
            http://json-smart.googlecode.com/svn/trunk/bench/src/net/minidev/bench/json/TestData.java
            for more details.

            > - why is JVM not warmed up -- by numbers it looks like numbers are
            > dominated by JVM/HotSpot runup times
            true, those bench include classPathLoading.
            I should add a warmed up benchmark.


            > - why not use one of existing frameworks to get at least some
            > semblance of due diligence wrt run times, variances and so on?

            TODO :)

            > I also agree in that it is wrong to call this json anything, given
            > that it is for data format that looks a bit like JSON; should be
            > called something else.

            By default Json-smart only using strict Json data, I only use compact json to fill my Mysql varchar(255) collumns.

            > And from this, comparing performance to real JSON parsers is somewhat
            > odd, since parsers actually work on different data formats.
            >
            > -+ Tatu +-

            Uriel


            --- In json@yahoogroups.com, Tatu Saloranta <tsaloranta@...> wrote:
            >
            > On Mon, May 16, 2011 at 3:16 AM, uriel.chemouni
            > <uriel.chemouni@...> wrote:
            > > Hi
            > >
            > > I have recently published my Json API.
            > >
            > > Features test:
            > > http://code.google.com/p/json-smart/wiki/FeaturesTests
            > >
            > > Benches test:
            > > http://code.google.com/p/json-smart/wiki/Benchmark
            > >
            > > Project home;
            > > http://code.google.com/p/json-smart
            > >
            > > Don't hesitate to send me your feedback.
            >
            > Performance test section could surely use some work. It tells very
            > little, but makes big unsubstantiated claims regarding performance.
            >
            > For example:
            >
            > - where is source?
            > - JSON used?
            > - why is JVM not warmed up -- by numbers it looks like numbers are
            > dominated by JVM/HotSpot runup times
            > - why not use one of existing frameworks to get at least some
            > semblance of due diligence wrt run times, variances and so on?
            >
            > I also agree in that it is wrong to call this json anything, given
            > that it is for data format that looks a bit like JSON; should be
            > called something else.
            > And from this, comparing performance to real JSON parsers is somewhat
            > odd, since parsers actually work on different data formats.
            >
            > -+ Tatu +-
            >
          • Tatu Saloranta
            On Mon, May 16, 2011 at 9:41 AM, uriel.chemouni ... Great, thanks! ... Yes, this makes big difference. Usually relative short amount of time works ok (like 5
            Message 5 of 18 , May 16 12:37 PM
            • 0 Attachment
              On Mon, May 16, 2011 at 9:41 AM, uriel.chemouni
              <uriel.chemouni@...> wrote:
              > Hi  Tatu,
              >> - where is source?
              >
              > I can publish (I'm not shure) a working copy of the benching software, due to dépendances licence.
              > However I have just upload the full benchmark package in the SVN of json-smart:
              > http://json-smart.googlecode.com/svn/trunk/bench/

              Great, thanks!

              ...
              >> - why is JVM not warmed up -- by numbers it looks like numbers are
              >> dominated by JVM/HotSpot runup times
              > true, those bench include classPathLoading.
              > I should add a warmed up benchmark.

              Yes, this makes big difference. Usually relative short amount of time
              works ok (like 5 seconds per test).

              >> - why not use one of existing frameworks to get at least some
              >> semblance of due diligence wrt run times, variances and so on?
              >
              > TODO :)

              :)

              One other thing that may be problematic is that input is given as a
              String: while I understand that this is sometimes useful, more often
              input comes as a byte stream (over the network, from file), and so
              step of converting byte input to String should actually be measured as
              its efficiency varies a lot between implementations: some implement
              efficient conversions whereas others rely on slower default
              mechanisms.
              This is why most JSON benchmarks (or general purpose JVM tests like
              https://github.com/eishay/jvm-serializers/wiki) hand a byte[] to
              deserializers.

              Good luck!

              -+ Tatu +-
            • Stephan Beal
              ... If you ll make the columns of type TEXT, the stored JSON can have any size. i use this in a wiki back-end which stores pages in JSON form. -- ...
              Message 6 of 18 , May 16 12:53 PM
              • 0 Attachment
                >
                > By default Json-smart only using strict Json data, I only use compact json
                > to fill my Mysql varchar(255) collumns.
                >

                If you'll make the columns of type TEXT, the stored JSON can have any size.
                i use this in a wiki back-end which stores pages in JSON form.

                --
                ----- stephan beal
                http://wanderinghorse.net/home/stephan/


                [Non-text portions of this message have been removed]
              • marsdenjw
                Hello Uriel, I think benchmarking is a great way to compare the frameworks (and motivate lazy developers like myself who never finish anything). I run the
                Message 7 of 18 , May 16 6:14 PM
                • 0 Attachment
                  Hello Uriel,

                  I think benchmarking is a great way to compare the frameworks (and motivate lazy developers like myself who never finish anything). I run the JSONiJ failed tests on the latest release and I get one fail and another which is open for discussion.

                  Unit test: http://pastebin.com/SjuzXxTb
                  Output: http://pastebin.com/4GYbHwKq

                  Specifically, it passes,

                  1. Support Empty Object
                  2. Support simple Object float value
                  3. Support lowcase float value
                  4. Array of empty Object

                  Its questionable if I pass
                  1. Bigint number support - I can see why this is an issue. I put it in that notation on purpose, but if its a key to something then it will break it.
                  Input: { "v":123456789123456789123456789}
                  Output: {"v":1.2345678912345679E26}

                  I totally fail
                  1. Double precision floating point

                  I will see if I can get your harness up and running and see if I can see why. At the moment i am working so it will have to wait.

                  John

                  --- In json@yahoogroups.com, "marsdenjw" <j.w.marsden@...> wrote:
                  >
                  > Wow, JSONiJ sucks.
                  >
                  > Regards,
                  >
                  > John (Developer, JSONiJ)
                  >
                  > --- In json@yahoogroups.com, "uriel.chemouni" <uriel.chemouni@> wrote:
                  > >
                  > > Hi
                  > >
                  > > I have recently published my Json API.
                  > >
                  > > Features test:
                  > > http://code.google.com/p/json-smart/wiki/FeaturesTests
                  > >
                  > > Benches test:
                  > > http://code.google.com/p/json-smart/wiki/Benchmark
                  > >
                  > > Project home;
                  > > http://code.google.com/p/json-smart
                  > >
                  > > Don't hesitate to send me your feedback.
                  > >
                  >
                • uriel.chemouni
                  This table is more than 150 000 000 lines long, switching to text will dramatically impact performances, and disk usage.
                  Message 8 of 18 , May 17 1:01 AM
                  • 0 Attachment
                    This table is more than 150 000 000 lines long, switching to text will dramatically impact performances, and disk usage.

                    --- In json@yahoogroups.com, Stephan Beal <sgbeal@...> wrote:
                    >
                    > >
                    > > By default Json-smart only using strict Json data, I only use compact json
                    > > to fill my Mysql varchar(255) collumns.
                    > >
                    >
                    > If you'll make the columns of type TEXT, the stored JSON can have any size.
                    > i use this in a wiki back-end which stores pages in JSON form.
                    >
                    > --
                    > ----- stephan beal
                    > http://wanderinghorse.net/home/stephan/
                    >
                    >
                    > [Non-text portions of this message have been removed]
                    >
                  • uriel.chemouni
                    Great, JSONiJ 0.2.3.9 can now validate all my tests. It deserves an update results in json-smart. Good job jhon.
                    Message 9 of 18 , May 20 10:35 PM
                    • 0 Attachment
                      Great, JSONiJ 0.2.3.9 can now validate all my tests.

                      It deserves an update results in json-smart.

                      Good job jhon.

                      --- In json@yahoogroups.com, "marsdenjw" <j.w.marsden@...> wrote:
                      >
                      > Hello Uriel,
                      >
                      > I think benchmarking is a great way to compare the frameworks (and motivate lazy developers like myself who never finish anything). I run the JSONiJ failed tests on the latest release and I get one fail and another which is open for discussion.
                      >
                      > Unit test: http://pastebin.com/SjuzXxTb
                      > Output: http://pastebin.com/4GYbHwKq
                      >
                      > Specifically, it passes,
                      >
                      > 1. Support Empty Object
                      > 2. Support simple Object float value
                      > 3. Support lowcase float value
                      > 4. Array of empty Object
                      >
                      > Its questionable if I pass
                      > 1. Bigint number support - I can see why this is an issue. I put it in that notation on purpose, but if its a key to something then it will break it.
                      > Input: { "v":123456789123456789123456789}
                      > Output: {"v":1.2345678912345679E26}
                      >
                      > I totally fail
                      > 1. Double precision floating point
                      >
                      > I will see if I can get your harness up and running and see if I can see why. At the moment i am working so it will have to wait.
                      >
                      > John
                      >
                      > --- In json@yahoogroups.com, "marsdenjw" <j.w.marsden@> wrote:
                      > >
                      > > Wow, JSONiJ sucks.
                      > >
                      > > Regards,
                      > >
                      > > John (Developer, JSONiJ)
                      > >
                      > > --- In json@yahoogroups.com, "uriel.chemouni" <uriel.chemouni@> wrote:
                      > > >
                      > > > Hi
                      > > >
                      > > > I have recently published my Json API.
                      > > >
                      > > > Features test:
                      > > > http://code.google.com/p/json-smart/wiki/FeaturesTests
                      > > >
                      > > > Benches test:
                      > > > http://code.google.com/p/json-smart/wiki/Benchmark
                      > > >
                      > > > Project home;
                      > > > http://code.google.com/p/json-smart
                      > > >
                      > > > Don't hesitate to send me your feedback.
                      > > >
                      > >
                      >
                    • marsdenjw
                      No problem - as I said. I think its a good way to compare frameworks. I am working on making it a little bit quicker now. It was built to parse input streams
                      Message 10 of 18 , May 21 3:47 AM
                      • 0 Attachment
                        No problem - as I said. I think its a good way to compare frameworks.

                        I am working on making it a little bit quicker now. It was built to parse input streams directly and not strings. The String current string parse is fine for small strings, but when they get large like your tests its a bit silly.

                        https://bitbucket.org/openecho/jsonij/src/653c6b5f85c8/src/main/java/jsonij/json/JSONParser.java#cl-90

                        It pushes the string contents into a byte buffer and then reads the bytes. Its not too much work to make another version of the reader itself to work directly on the input string.

                        John

                        --- In json@yahoogroups.com, "uriel.chemouni" <uriel.chemouni@...> wrote:
                        >
                        > Great, JSONiJ 0.2.3.9 can now validate all my tests.
                        >
                        > It deserves an update results in json-smart.
                        >
                        > Good job jhon.
                        >
                        > --- In json@yahoogroups.com, "marsdenjw" <j.w.marsden@> wrote:
                        > >
                        > > Hello Uriel,
                        > >
                        > > I think benchmarking is a great way to compare the frameworks (and motivate lazy developers like myself who never finish anything). I run the JSONiJ failed tests on the latest release and I get one fail and another which is open for discussion.
                        > >
                        > > Unit test: http://pastebin.com/SjuzXxTb
                        > > Output: http://pastebin.com/4GYbHwKq
                        > >
                        > > Specifically, it passes,
                        > >
                        > > 1. Support Empty Object
                        > > 2. Support simple Object float value
                        > > 3. Support lowcase float value
                        > > 4. Array of empty Object
                        > >
                        > > Its questionable if I pass
                        > > 1. Bigint number support - I can see why this is an issue. I put it in that notation on purpose, but if its a key to something then it will break it.
                        > > Input: { "v":123456789123456789123456789}
                        > > Output: {"v":1.2345678912345679E26}
                        > >
                        > > I totally fail
                        > > 1. Double precision floating point
                        > >
                        > > I will see if I can get your harness up and running and see if I can see why. At the moment i am working so it will have to wait.
                        > >
                        > > John
                        > >
                        > > --- In json@yahoogroups.com, "marsdenjw" <j.w.marsden@> wrote:
                        > > >
                        > > > Wow, JSONiJ sucks.
                        > > >
                        > > > Regards,
                        > > >
                        > > > John (Developer, JSONiJ)
                        > > >
                        > > > --- In json@yahoogroups.com, "uriel.chemouni" <uriel.chemouni@> wrote:
                        > > > >
                        > > > > Hi
                        > > > >
                        > > > > I have recently published my Json API.
                        > > > >
                        > > > > Features test:
                        > > > > http://code.google.com/p/json-smart/wiki/FeaturesTests
                        > > > >
                        > > > > Benches test:
                        > > > > http://code.google.com/p/json-smart/wiki/Benchmark
                        > > > >
                        > > > > Project home;
                        > > > > http://code.google.com/p/json-smart
                        > > > >
                        > > > > Don't hesitate to send me your feedback.
                        > > > >
                        > > >
                        > >
                        >
                      • Tatu Saloranta
                        ... Are these tests also available with source code? Results for Jackson looked odd; as if some assumptions were being made that are not true (assuming
                        Message 11 of 18 , May 21 9:20 AM
                        • 0 Attachment
                          On Sat, May 21, 2011 at 3:47 AM, marsdenjw <j.w.marsden@...> wrote:
                          > No problem - as I said. I think its a good way to compare frameworks.

                          Are these tests also available with source code? Results for Jackson
                          looked odd; as if some assumptions were being made that are not true
                          (assuming round-tripping of white space, or specific style of escaping
                          characters)

                          >
                          > I am working on making it a little bit quicker now. It was built to parse input streams directly and not strings. The String current string parse is fine for small strings, but when they get large like your tests its a bit silly.

                          I think using String as input does not make much sense -- most
                          real-world use cases start with a byte sequence (reading from a file,
                          receiving or sending requests), and it is only question of who does
                          decoding/encoding necessary. When starting with a String (or Reader)
                          it is assumed that parser can not do this, which is true for some
                          parsers but not all. And since encoding/decoding between bytes and
                          characters is a major factor in performance, this skews results a lot.

                          -+ Tatu +-
                        • marsdenjw
                          Hi Tatu, If that question was addressed to me, then no, I dont have the source. I just made unit tests for each test I was said to fail and went from there
                          Message 12 of 18 , May 23 3:49 AM
                          • 0 Attachment
                            Hi Tatu,

                            If that question was addressed to me, then no, I dont have the source. I just made unit tests for each test I was said to fail and went from there (https://bitbucket.org/openecho/jsonij/src/a25fd28c1d77/src/test/java/jsonij/json/SJTest.java). Im not sure how fussy his code is about spaces and differences in valid results...

                            I agree totally with the byte sequence comment. Every time I use JSON I am parsing from a HTTP source or I am loading from a file.

                            Regards,

                            John

                            --- In json@yahoogroups.com, Tatu Saloranta <tsaloranta@...> wrote:
                            >
                            > On Sat, May 21, 2011 at 3:47 AM, marsdenjw <j.w.marsden@...> wrote:
                            > > No problem - as I said. I think its a good way to compare frameworks.
                            >
                            > Are these tests also available with source code? Results for Jackson
                            > looked odd; as if some assumptions were being made that are not true
                            > (assuming round-tripping of white space, or specific style of escaping
                            > characters)
                            >
                            > >
                            > > I am working on making it a little bit quicker now. It was built to parse input streams directly and not strings. The String current string parse is fine for small strings, but when they get large like your tests its a bit silly.
                            >
                            > I think using String as input does not make much sense -- most
                            > real-world use cases start with a byte sequence (reading from a file,
                            > receiving or sending requests), and it is only question of who does
                            > decoding/encoding necessary. When starting with a String (or Reader)
                            > it is assumed that parser can not do this, which is true for some
                            > parsers but not all. And since encoding/decoding between bytes and
                            > characters is a major factor in performance, this skews results a lot.
                            >
                            > -+ Tatu +-
                            >
                          • Tatu Saloranta
                            ... Sorry, it was question to author, and he did actually add a link to wiki. So I was able to figure it out, and submit a patch for jackson test. I hope
                            Message 13 of 18 , May 23 10:42 AM
                            • 0 Attachment
                              On Mon, May 23, 2011 at 3:49 AM, marsdenjw <j.w.marsden@...> wrote:
                              > Hi Tatu,
                              >
                              > If that question was addressed to me, then no, I dont have the source. I just made unit tests for each test I was said to fail and went from there (https://bitbucket.org/openecho/jsonij/src/a25fd28c1d77/src/test/java/jsonij/json/SJTest.java). Im not sure how fussy his code is about spaces and differences in valid results...

                              Sorry, it was question to author, and he did actually add a link to
                              wiki. So I was able to figure it out, and submit a patch for jackson
                              test. I hope authors of other JSON libs can do the same, so that
                              results reflect actual working of libs.

                              > I agree totally with the byte sequence comment. Every time I use JSON I am parsing from a HTTP source or I am loading from a file.

                              Yeah, that is my experience as well. Maybe this was based on using a
                              framework that does loading and decoding first, just passing JSON
                              contents. Occasionally this is true, when binding data from form
                              parameters or something. But not very often at least for me.

                              -+ Tatu +-
                            • uriel.chemouni
                              do not hesitate to submit paths to my benchmark code. The new version display warmed-up JVM result, and display all results as percents. The results are easier
                              Message 14 of 18 , May 23 10:49 AM
                              • 0 Attachment
                                do not hesitate to submit paths to my benchmark code.
                                The new version display warmed-up JVM result, and display all results as percents.
                                The results are easier to read.

                                --- In json@yahoogroups.com, Tatu Saloranta <tsaloranta@...> wrote:
                                >
                                > On Mon, May 23, 2011 at 3:49 AM, marsdenjw <j.w.marsden@...> wrote:
                                > > Hi Tatu,
                                > >
                                > > If that question was addressed to me, then no, I dont have the source. I just made unit tests for each test I was said to fail and went from there (https://bitbucket.org/openecho/jsonij/src/a25fd28c1d77/src/test/java/jsonij/json/SJTest.java). Im not sure how fussy his code is about spaces and differences in valid results...
                                >
                                > Sorry, it was question to author, and he did actually add a link to
                                > wiki. So I was able to figure it out, and submit a patch for jackson
                                > test. I hope authors of other JSON libs can do the same, so that
                                > results reflect actual working of libs.
                                >
                                > > I agree totally with the byte sequence comment. Every time I use JSON I am parsing from a HTTP source or I am loading from a file.
                                >
                                > Yeah, that is my experience as well. Maybe this was based on using a
                                > framework that does loading and decoding first, just passing JSON
                                > contents. Occasionally this is true, when binding data from form
                                > parameters or something. But not very often at least for me.
                                >
                                > -+ Tatu +-
                                >
                              • Mark Joseph
                                https://www.p6r.com/articles/2012/01/30/p6r-kmip-toolkit-introduction/ Best, Mark Joseph, Ph.D. President P6R, Inc 408-205-0361 mark@p6r.com Skype:
                                Message 15 of 18 , Feb 1, 2012
                                • 0 Attachment
                                  https://www.p6r.com/articles/2012/01/30/p6r-kmip-toolkit-introduction/



                                  Best,

                                  Mark Joseph, Ph.D.
                                  President
                                  P6R, Inc
                                  408-205-0361
                                  mark@...
                                  Skype: markjoseph_sc
                                  _____



                                  [Non-text portions of this message have been removed]
                                Your message has been successfully submitted and would be delivered to recipients shortly.