Replies: 2 comments 1 reply
-
Long story short; when trying to debug real world problems with transferring data from one system to another, think different CPU architecture, OS, programming language, etc. there is no single debugger that can help you find the problem. Yes you can typically put a debugger on either end but I know of nothing that looks at both ends for all the different protocols simultaneously on both computers. Now if the data is on a real world production system and is production data that can not be put on a dev machine and the production system is not allowed to have development tools on it, how do you identify the problem? The way to find the problems is to do it the old fashioned way and look at the data and hand walk the spec and the code. As such over the years people doing such work prefer that the data in the pipe be human readable and that there be a very rigid spec to eliminate possible ambiguities when creating the data to be put into the pipe. In the old days making the data human readable was unthinkable because of the memory constraints and time constraints. Think of the days when computers had memory in the KBs and transmission speeds were in the kilobits per second. The more compressed the data the better, but even then compression algorithms were not a common standard. But now there is much more memory and faster transmission speeds so making it human readable is more advantageous. Also compressing the human readable data going into the pipe and uncompressing the data coming out of the pipe is now much more standardized on a few compression algorithms. When you take into account the history and legacy of this it really does make sense as to where we are and how we arrived here. |
Beta Was this translation helpful? Give feedback.
-
In an earlier version of the original post that @EricGT responded to, I presented some wacky ideas about drawing a hard line between serialization and schema transformation. I've since realized these ideas don't make sense and aren't even necessary to consider when dealing with JSON schemas. I've edited the proposed interface in the original post to be much simpler and more intuitive. |
Beta Was this translation helpful? Give feedback.
-
EDIT - PLEASE IGNORE THIS POST: My thinking on schemas has evolved a lot since making this post, but I'm not ready to propose any new ideas yet. Please ignore this post.
I'm trying to come up with the best way to reason about schemas in Prolog in order to implement JSON Schema as a Scryer library. Please let me know whether my current thinking below is on the right track.
I'm not happy with definitions of either serialization or schema that I've seen online, so I will define them in Prolog terms myself.
Serialization and schema can be treated independently. There's no reason why a JSON Schema can't describe the contents of a CSV file, or a Protobuf message definition can't describe the contents of an Apache Avro binary encoded message.
The below three predicates will be the interfaces for JSON serialization/deserialization both with and without schemas.
json_chars//1
(public) serializes and deserializes Prolog terms to and from JSON with full compliance to the official spec.json_schema/3
(public) will relate three Prolog terms:json_chars//1
or some other sourceschema.json
file usingjson_chars//1
or constructed manually.json_chars//2
(public) will be a wrapper around the above two predicates for serializing/deserializing JSON with an explicit schema. It will be defined as something like this:How this interface will work with some use cases:
library(json)
andlibrary(csv)
deserialize data into the exact same format of Prolog term (which is not currently the case), it would be possible to answer queries like "Can this data I got from a CSV file be serialized as valid JSON?" and "Does this CSV data conform to this particular JSON Schema?"Beta Was this translation helpful? Give feedback.
All reactions