NeoJSON.pillar

{
    "metadata" : {
        "title": "NeoJSON",
        "attribution": "Sven Van Caekenberghe with Damien Cassou and Stéphane Ducasse"
    },
    "headingLevelOffset":2
}
@cha:JSON


JSON (JavaScript Object Notation) is a popular data-interchange format. NeoJSON is an elegant and efficient standalone Smalltalk library to read and write
JSON converting to and from Smalltalk objects. The library is developed and actively maintained by Sven Van Caekenberghe.

!An Introduction to JSON

JSON is a lightweight text-based open standard designed for human-readable data interchange. It was derived from the JavaScript scripting language for
representing simple data structures and associative arrays, called objects. Despite its relationship to JavaScript, it is language independent, with parsers
available for many languages.

@@note References: *http://www.json.org/*, *http://en.wikipedia.org/wiki/Json* and *http://www.ietf.org/rfc/rfc4627.txt?number=4627*.

There are only a couple of primitive types in JSON:

-numbers (integer or floating point)
-strings
-the boolean constants ==true== and ==false==
- ==null==

Only two composite types exist:

-lists (an ordered sequenece of values)
-maps (an unordered associative array, mapping string property names to values)

That is really all there is to it. No options or additions are defined in the standard.

! NeoJSON

To load NeoJSON, evaluate the following:

[[[
Gofer it
   smalltalkhubUser: 'SvenVanCaekenberghe' project: 'Neo';
   configurationOf: 'NeoJSON';
   loadStable.
]]]


The NeoJSON library contains a reader (the class ==NeoJSONReader==) and a writer (the class ==NeoJSONWriter==) to parse, respectively generate, JSON to and from Pharo objects. The goals of NeoJSON are:

-to be standalone (have no dependencies and little requirements);
-to be small, elegant and understandable;
-to be efficient (both in time and space);
-to be flexible and non-intrusive.

Compared to other Smalltalk JSON libraries, NeoJSON

-has less dependencies and little requirements;
-can be more efficient (be faster and use less memory);
-allows for the use of schemas and mappings.

!Primitives

Obviously, the primitive types are mapped to corresponding Pharo classes. While reading:

- JSON numbers become instances of ==Integer== or ==Float==
- JSON strings become instances of ==String==
- JSON booleans become instances of ==Boolean==
- JSON ==null== becomes ==nil==

While writing:

- Pharo numbers are converted to floats, except for instances of ==Integer== that become JSON integers
- Pharo strings become JSON strings
- Pharo booleans become JSON booleans
- Pharo ==nil== becomes JSON ==null==


! Generic Mode

NeoJSON can operate in a generic mode that requires no further configuration.

!! Reading from JSON

While reading:

-JSON maps become instances of mapClass, ==Dictionary== by default;
-JSON lists become instances of listClass, ==Array== by default.

The following example creates a Pharo array from a JSON expression:

[[[
NeoJSONReader fromString: ' [ 1,2,3 ] '.
]]]

This expression can be decomposed to better control the reading process:

[[[
(NeoJSONReader on: ' [ 1,2,3 ] ' readStream)
   listClass: OrderedCollection;
   next.
]]]

The above expression is equivalent to the previous one except that a Pharo ordered collection will be used in place of an array.


The next example creates a Pharo dictionary (with =='x'== and =='y'== keys):

[[[
NeoJSONReader fromString: ' { "x" : 1, "y" : 2 } '.
]]]

To automatically convert keys to symbols, pass ==true== to ==propertyNamesAsSymbols:== like this:

[[[
(NeoJSONReader on: ' { "x" : 1, "y" : 2 } ' readStream)
   propertyNamesAsSymbols: true;
   next
]]]

The result of this expression is a dictionary with ==#x== and ==#y== as keys.

!!Writing to JSON

While writing:

-instances of ==Dictionary== and ==SmallDictionary== become maps;
-all other collections become lists;
-all other non-primitive objects are rejected.

Here are some examples writing in generic mode:

[[[
NeoJSONWriter toString: #(1 2 3).
NeoJSONWriter toString: { Float pi. true. false. 'string' }.
NeoJSONWriter toString: { #a -> '1' . #b -> '2' } asDictionary.
]]]

Above expressions return a compact string (''i.e.'', with neither indentation nor new lines). To get a nicely formatted output, use ==toStringPretty:== like this:

[[[
NeoJSONWriter toStringPretty: #(1 2 3).
]]]

In order to use the generic mode, you have to convert your domain objects to and from ==Dictionary== and ==SequenceableCollection==. This is relatively easy but not very efficient, depending on the use case.


! Schemas and Mappings

NeoJSON allows for the optional specification of schemas and mappings to be used when writing or reading.

When writing, mappings are used when arbitrary objects are seen. For example, in order to write an array of points, you could do as follows:

[[[
String streamContents: [ :stream |
   (NeoJSONWriter on: stream)
      prettyPrint: true;
      mapInstVarsFor: Point;
      nextPut: (Array with: 1@3 with: -1@3) ].
]]]

Collections are handled automatically, like in the generic case. As a result, the above expression returns a string containing:

[[[language=json
[
   {
      "x" : 1,
      "y" : 3
   },
   {
      "x" : -1,
      "y" : 3
   }
]
]]]

When reading, a mapping is used to specify what Pharo object to instantiate and how to instantiate it. Here is a very simple case, reading a map as a point:

[[[
(NeoJSONReader on: ' { "x" : 1, "y" : 2 } ' readStream)
   mapInstVarsFor: Point;
   nextAs: Point.
]]]

Since JSON lacks a universal way to specify the class of an object, we have to specify the target schema that we want to use as an argument to ==nextAs:==.

To define the schema of the elements in a list, write something like the following:

[[[
(NeoJSONReader
      on: ' [{ "x" : 1, "y" : 2 },
             { "x" : 3, "y" : 4 }] ' readStream)
   mapInstVarsFor: Point;
   for: #ArrayOfPoints
      customDo: [ :mapping | mapping listOfElementSchema: Point ];
   nextAs: #ArrayOfPoints.
]]]

The above expression returns an array of 2 points. As you can see, the argument to ==nextAs:== can be a class (as seen previously) or any symbol, provided the mapper knows about it.

To get an ==OrderedCollection== instead of an array as output, you should use the ==listOfType:== message:

[[[
(NeoJSONReader on: ' [ 1, 2 ] ' readStream)
   for: #Collection
      customDo: [ :mapping | mapping listOfType: OrderedCollection ];
   nextAs: #Collection.
]]]

To specify how values in a map should be instantiated, use the ==mapWithValueSchema:==:

[[[
(NeoJSONReader on: ' { "point1" : {"x" : 1, "y" : 2 } }' readStream)
   mapInstVarsFor: Point;
   for: #DictionaryOfPoints
      customDo: [ :mapping | mapping mapWithValueSchema: Point ];
   nextAs: #DictionaryOfPoints.
]]]

The above expression returns a ==Dictionary== with 1 key-value pair =='point1' -> (1@2)==.

Working with nested types is easy when generating JSON but a bit more work when parsing. 
A Rectangle contains Points in its instance variables. 
Here is how to generate JSON for a Rectangle.

[[[
String streamContents: [ :stream | 
   (NeoJSONWriter on: stream)
      prettyPrint: true;
      mapInstVarsFor: Point;
      mapInstVarsFor: Rectangle;
      nextPut: (Rectangle origin: 3 @ 4 extent: 5 @ 6)
]]]

Which gives the following output.

[[[
{
   "origin" : {
      "x" : 3,
      "y" : 4
   },
   "corner" : {
      "x" : 8,
      "y" : 10
   }
}
]]]

In most cases, you just map all instance variables for each class that you encounter.

As you can see all typing information is gone from the JSON expression.
There is no way to know we are looking at a Rectangle with 2 embedded Points.
Hence we have to provide this information when parsing, using mappings.
We assume ==rectangleJson== contains the JSON output generated above as a String.

[[[
(NeoJSONReader on: rectangleJson readStream)
   mapInstVarsFor: Point;
   for: Rectangle do: [ :mapping | 
      (mapping mapInstVar: #origin) valueSchema: Point.
      (mapping mapInstVar: #corner) valueSchema: Point ];
   nextAs: Rectangle
]]]

Again we map all instances variables for both Point and Rectangle, but with a twist.
We have to specify the ==valueSchema== or type of the instance variables
==origin== and ==corner== of Rectangle to be Points.
Another way to do the same thing is as follows.

[[[
(NeoJSONReader on: rectangleJson readStream)
   mapInstVarsFor: Point;
   for: Rectangle do: [ :mapping | 
      mapping mapInstVars do: [ :each | each valueSchema: Point ] ];
   nextAs: Rectangle
]]]

Here we take advantage of the fact that all of Rectangle's instance variables are of the same type.

You can go beyond pre-defined messages and specify a decoding block:

[[[
(NeoJSONReader on: ' "2015/06/19" ' readStream)
   for: DateAndTime
      customDo: [ :mapping |
         mapping decoder: [ :string |
            DateAndTime fromString: string ] ];
   nextAs: DateAndTime.
]]]

The above expression returns an instance of ==DateAndTime==. The message ==encoder:== is used to do the opposite, ''i.e.'' convert from a Smalltalk object to JSON:

[[[
String streamContents: [ :stream |
   (NeoJSONWriter on: stream)
      for: DateAndTime
         customDo: [ :mapping | mapping encoder: #printString ];
      nextPut: DateAndTime now ].
]]]

The above expression returns a string representing the current date and time.

NeoJSON deals efficiently with mappings: the minimal amount of intermediary structures are created.
On modern hardware, NeoJSON can write or read tens of thousands of small objects per second. Several benchmarks are included in the unit tests package.

!Emitting null Values

For efficiency reasons, by default, ==NeoJSONWriter== does not write ==nil== values:

[[[
String streamContents: [ :stream |
   (NeoJSONWriter on: stream)
      mapAllInstVarsFor: Point;
      nextPut: Point new ].
]]]

The above expression returns the =='{}'== string. If you want to see the uninitialized instance properties, pass ==true== to the ==writeNil:== message:

[[[
String streamContents: [ :stream |
   (NeoJSONWriter on: stream)
      mapAllInstVarsFor: Point;
      writeNil: true;
      nextPut: Point new ].
]]]

The above expression returns the =='{"x":null,"y":null}'== string.

! Conclusion

NeoJSON is a powerful library to convert objects. Sven, the author of NeoJSON, also developed STON (Smalltalk object notation) which is closer to Pharo syntax and handles cycles and references between serialized objects.

%  LocalWords:  Caekenberghe

% Local Variables:
% compile-command: "cd .. && ./pillar export --to='LaTeX by chapter' NeoJSON/NeoJSON.pillar && bash pillarPostExport.sh"
% End: