Thursday, March 20, 2014

A JSON Library Evaluation for Java EE applications

I have a need to enhance a set of Java EE applications to support mobile development projects.   As mobile developers seem to prefer JSON formats to XML for passing data to/from mobile devices, I had a need evaluate current Java JSON libraries for use by the supporting Java EE applications.  The client prefers open source products to vended products.  The most prevalent open source product choices seem to be the following:

This evaluation was conducted in March, 2014.  The evaluation was conducted using the methodology described in chapter 13 of the Java EE Architect's Handbook.

Evaluation Criteria

As with all product evaluations, it’s important to establish the criteria by which product choices will be graded and on which a product decision will be made.  The criteria used in such a product choice would obviously vary per project and organization.  I’ve settled on the following criteria:

  • Level of community activity - measured by the average number of releases per year.  Active projects are more likely to be enhanced in future.  Should the product not be maintained and become obsolete, it may need to be replaced in future and may incur additional costs of ownership.
  • Market share – measured by the number of downloads as a proxy.  I would prefer direct market share information instead of a proxy, but that isn’t usually available for open source products. Solutions for problems and issues with the product are more likely to be posted on the web for more popular products; such increased posting speeds development and indirectly lowers cost of ownership.  
  • License – The license for the product must conform to a specified list of legal requirements; most common open source licenses are acceptable.
  • Ease of Use – subjective measurement based on code samples and the quality of documentation.   Ease of use features greatly speed development and indirectly lowers  the cost of ownership.  
  • Performance – measured by speed and required memory footprint.  Lower footprints lower hardware requirements and lowers cost of ownership.   Same test case used for all products so that performance can be more easily compared.
  • Versioning Support – I expect that as features are added, existing JSON formats will be changed and enhanced.  With a mobile development with the application distributed to the public, forcing upgrades to accommodate JSON format changes isn’t feasible.  A JSON library that provides support for legacy formats is preferred.  it’s possible that a versioning solution will involve additional products and not solely reside within the JSON library itself.
Not all criteria are necessarily weighted equally; weights given to each item will vary per project and enterprise.  It’s also possible that individual projects might have additional criteria not listed here.

Evaluation Results and Product Ranking

The evaluation criteria were prioritized by the needs of my upcoming project.  It's possible that your priorities might be different.  All criteria were assigned a numerical score from one to ten with ten being the best.

Scores for the criteria were weighted by priority.  High priority scores were multiplied by 2.  Medium priority scores were multiplied by 1.5.  Low priority scores were left alone.  Weighted scores for all criteria were then added together for the overall ranking.  Reasons for the ratings given are discussed in more detail in the product observations section below.


Product Observations


Ease of Use

I view this as the most important of the criteria as it most directly lower costs of ownership.  With both Gson and Jackson creating and reading JSON formatted data were two liners.  I’ve provided the source for my prototypes in the reference section below.  With my prototype, there was no need to write custom serialization/deserialization logic; both products do have robust options to do this should it be needed for your projects.  Both products by default handled escaping special characters like quotes and carriage returns.  The prototype source code and all dependencies can be downloaded from here.

One reason Gson slightly edges out Jackson is because it has better documentation.  The Gson documentation is better organized and more concise.  As a result of that documentation, constructing the Gson prototype took a little less time than writing the Jackson prototype.
It’s also notable that Jackson has a clumsy distribution: They need a one-zip download that contains binary jars, source jars, Javadoc jars, license and documentation.  Instead, all of these are downloaded separately and the product is structured so that it’s not obvious which jars you need for your particular project.

Note that coding using the classes provided with the JSR 353 spec takes much longer and it’s much easier to create bugs.  My prototype (I re-coded the same product with each of the products) required over 100 lines of code using the JSR 353 construct as opposed to the handful required by either Gson or Jackson.

I did not code a prototype for the Json.org product as it’s not a real product.  There’s no formal distribution bundle.  For distribution, you download class source individually and build it.  There are no unit tests, so you can’t validate your build very easily.  Essentially, product source becomes additional source in your project that you must maintain.  It became clear that this product isn’t a valid option, despite the fact it appeared near the top of internet searches for JSon libraries.

Level of Community Activity

Community activity is important for open source projects as products will become obsolete without care and feeding.  Being part of the Java EE specification means that the JSR353 specification have or will have support from all the Java EE implementers, both commercial and open source.  It’s hard to compete with that.

It should be noted that both Gson and Jackson typically have multiple releases per year.  Both products are used in several open source products.  Both Gson and Jackson have ample community support and will no doubt be enhanced for some time to come.

Market Share

The number of downloads could only available for one of the four products:  Gson had 143,904 downloads as of March 14, 2014.  Somebody built the binaries for the java classes on Json.org; that unofficial distribution had 10,821 downloads on March 14, 2014.  Download statistics for Jackson isn’t published as far as I can find.  For JSR353, it’s not possible to identify download statistics; it’s not clear that everybody who downloads a Java EE application server with JSR 353 support in it will use that section of the container for JSON processing; they could use either GSon, Jackson, or some other library.

Performance

The performance test was identical for all products.  The test consisted of starting with a complex value object with data and using that object to produce JSON-formatted data.  That JSON formatted data was also read and marshaled into the value objects.  This is a very common usage scenario.  Source for the performance evaluation can be downloaded from here.

Memory was measured by noting consumed memory (total memory less free memory) both before and after the test.  The test was run for 100,000 iterations to make time and memory usage more noticeable.  Results are in the table below.

The JSR353 reference implementation is at the extreme; it was a lot slower than the other two libraries, but appears to have a much smaller footprint.  It’s possible that the JSR test took so long that there was a garbage collection during the test, which means that the memory measurement is understated.

Comparing Gson and Jackson, Gson was faster and had a smaller memory footprint for reading Json data and marshalling that data into value objects.  Jackson was faster and had a smaller memory footprint producing Json data from value objects.

Versioning Support

Gson versioning support uses annotations.   When coding value objects that will correspond to the JSON data read or produced, you use the @Since annotation t record the version in which the field appeared.  When you instantiate Gson to read or produce JSON data, you have the option to specify the version of JSON that will be used.  Any fields from later versions will be ignored.

None of the other products appear to have versioning support.

License.

All three viable product options have licenses that are acceptable to most organizations.  Gson uses Apache 2.0.  Jackson’s license appears to have changed over time; Jackson was LGPL up to version 2.1 and an Apache license thereafter.  Jackson does not specify clearly which version of the Apache license it uses.  Furthermore, its distribution method doesn’t place a copy of the license in what you download.

Concluding Thoughts

Either Gson or Jackson are reasonable choices.  Both products are very easy to use for common use cases and have options for customized serialization/deserialization logic if needed; for most uses custom logic won’t be needed.

My choice for my upcoming project is Gson.  I’m lured by the superior documentation and versioning support.

I hope you’ve found this entry helpful.  Thanks for taking time to read it.

References


5 comments:

  1. Also see https://github.com/RichardHightower/json-parsers-benchmark for an upcoming alternative.

    ReplyDelete
  2. Thanks for the suggestion, the benchmark looks interesting.

    ReplyDelete
  3. One minor nit to pick: I think that the evaluation of community participation is invalid: process with JSR/JCP may be relevant wrt specification, but after 1.0 was finalized, there has been very little, if any, community involvement outside of Oracle. In contrast, as you note, community activity for Gson and Jackson has been steady for 5+ years, both regarding bug reporting and feedback and active contributions. This is related to usage, market share obviously, and jsr-353 not having much adoption does not help. But even bigger challenge is Oracle's ambivalent attitude towards open-source, collaboration and so on.

    In that regard, maybe it'd be better to at least separate different aspects; open standardization (for which one may consider JSR/JCP significant), and then active community participation of actual implementation (JSR/JCP is, after all, only for spec, and possibly -- but not necessarily -- for ref.impl).

    ReplyDelete
  4. For a while I've wanted a model versioning feature in Jackson, so I recently sat down and wrote one. Hopefully this helps any Googlers who stumble in here.

    https://github.com/jonpeterson/jackson-module-model-versioning

    ReplyDelete