- Johanna Antonelli
-
Joined JUXT as intern, full time year later after masters
-
Tutorial was first task
-
The space tutorial is designed as a gentle introduction to Crux.
-
Using basic Clojure
-
Choose your own adventure.. (debatable)
-
made to be a welcoming introduction to Crux
-
gentle introduction to basic concepts
-
basic clojure used
Technical questions at the end
-
The tutorial aims to cover topics as standalone as can be. This means a lot of questions may be answered as we progress.
-
Make a note.
Welcome to the year 2115
Congratulations on your new position at Helios Banking Inc.
-
Require juxt/crux as a dependency in your project
-
It’s the year 2115.
-
You have been hired by an inter-planetary bank, Helios Banking Inc.
-
Your task is to travel around the solar system completing assignments using Crux.
-
You have been given a Crux manual which will help you along your way.
-
In preparation you will have set up a project with crux as a dependency
-
instructions in the posts.
A warm welcome from the Helios family.
For your first assignment we would like you to go to Pluto and help Tombaugh Resources Ltd. set up their stock reporting system.
Don’t forget to fill in your flight manifest, else you won’t be permitted entry to the planets.
Before you leave you must fill in your flight manifest. To do this, you must first set up a Crux node.
For a Clojure in-process node, the start-node function accepts a module tree, a file, or a resource.
(require '[crux.api :as crux]
'[clojure.java.io :as io])
(crux/start-node {...}) ; module tree
(crux/start-node (io/file ...))
(crux/start-node (io/resource ...))(require '[crux.api :as crux]
'[clojure.java.io :as io])
(def node
(crux/start-node
{:crux.node/topology '[crux.standalone/topology]
:crux.kv/db-dir "data/db-dir"}))-
You want to get started as quickly as possible so you decide to use Crux’s inbuilt standalone node topology.
-
There is no persistence with this, so it is perfect for testing and debug.
-
Even so, Crux needs a database directory specified
You take a look around your ship and check the critical levels.
You read the manual entry for putting data into Crux.
Crux takes data in document form.
Each document must be in Extensible Data Notation (edn) and each document must contain a unique :crux.db/id value.
However, beyond those two requirements you have the flexibility to add whatever you like to your documents because Crux is schemaless.
(def manifest
{:crux.db/id :manifest
:pilot-name "Johanna"
:id/rocket "SB002-sol"
:id/employee "22910x2"
:badges "SETUP"
:cargo ["stereo" "gold fish" "slippers" "secret note"]})-
Just as you’re about to write your manifest, one of the porters passes you a secret note and asks you to deliver it to a martian named Kaarlang.
-
They are certain you will meet Kaarlang on your travels and so you see no harm in delivering the note for them.
(crux/submit-tx node [[:crux.tx/put manifest]])
;;=>#:crux.tx{:tx-id 0, :tx-time #inst "2020-06-18T13:54:08.375-00:00"}This is put, one of Crux’s four transaction operations.
Check that this was successful by asking Crux to show the entity history.
(crux/entity-history (crux/db node) :manifest :asc)
;;=> {:crux.tx/tx-time #inst "2020-06-18T13:54:08.375-00:00",
;; :crux.tx/tx-id 0,
;; :crux.db/valid-time #inst "2020-06-18T13:54:08.375-00:00",
;; :crux.db/content-hash #crux/id "0ab888b62775eea2eb2fffe10c9f6bfbf661a792"}-
Which returns the history of the provided entity id at this point in time.
-
You can use
:ascor:descflag with entity-history for the ascending or descending history respectively.
-
As you enter the Plutonian atmosphere, a message pops up on your communication panel:
Welcome to the dwarf planet Pluto. You are entering privately governed space. If you do not have the correct papers, entry will be denied. We hope you enjoy your stay.
Have a nice day.
— Anarchic Directorate of Pluto
The government of Pluto is asking to see your flight manifest.
| Transaction | Description |
|---|---|
|
Writes a version of a document |
|
Deletes a version of a document |
|
Stops a transaction if the precondition is not met. |
|
Removes a document entirely |
Put:
The put operation is used to write versions of a document (doc).
Each document must be in Extensible Data Notation (edn) and must contain a unique :crux.db/id value.
However, beyond those two requirements you have the flexibility to add whatever you like to your documents because Crux is schemaless.
Along with the document (doc), put has two optional additional arguments:
start valid-time
|
The time at which the entry will be valid from. |
end valid-time
|
The time at which the entry will be valid until. |
This means that you can query back through Crux, you can use valid-time arguments to see the state of Crux at a different time.
If no valid-time is provided, start valid-time defaults to the transaction time and the document is valid either eternally or until the document is updated by a future transaction with the same eid.
- Time in Crux is denoted
-
#inst "yyyy-MM-ddThh:mm:SS" - For example, 9:30 pm on January 2nd 1999 would be written:
-
#inst "1999-01-02T21:30:00". - A complete
puttransaction has the form -
[:crux.tx/put doc valid-time-start valid-time-end]
-
Currently there are only four transaction operations in Crux:
put,delete,matchandevict.
| Task |
Commodity Logging |
| Company |
Tombaugh Resources Ltd. |
| Contact |
R. Glogofloon |
| Submitted |
2115-02-20T13:38:20 |
| Additional information |
We need help setting up a new recording system for our mine. I have enclosed a list of the commodities we deal with. Please send someone soon because we already have a week’s worth of unrecorded stock-takes. |
| Attachments |
You land on the surface of the dwarf planet. As you do, the job ticket for this assignment is unlocked.
(crux/submit-tx node
[[:crux.tx/put
{:crux.db/id :commodity/Pu
:common-name "Plutonium"
:type :element/metal
:density 19.816
:radioactive true}]
[:crux.tx/put
{:crux.db/id :commodity/N
:common-name "Nitrogen"
:type :element/gas
:density 1.2506
:radioactive false}]
[:crux.tx/put
{:crux.db/id :commodity/CH4
:common-name "Methane"
:type :molecule/gas
:density 0.717
:radioactive false}]])
;;=> #:crux.tx{:tx-id 0, :tx-time #inst "2020-06-18T14:11:51.087-00:00"}-
You make your way over to the mines on the next shuttle.
-
On your way you decide to get a head start and put the commodities into Crux.
-
Since it takes six hours for each transaction to reach your Crux node on Earth from here, it is a good idea to batch up all the commodities in a single transaction.
Hello, I’m glad you’re here.
I would like you to fill in our last week’s worth of data on our commodities. We need to be able to look back at a given day and see what our stocks were for auditing purposes.
The stock for each day must be submitted at 6pm Earth time (UTC) for your banks records.
Are you able to do that for me?
— R. Glogofloon
-
Reginald is asking if we can fill in the last weeks worth of data for their stocks.
-
He’s given us the valid time it needs to go in at for auditing purposes
crux/submit-tx node
[[:crux.tx/put
{:crux.db/id :stock/Pu
:commod :commodity/Pu
:weight-ton 21 }
#inst "2115-02-13T18"] ;; valid-time
[:crux.tx/put
{:crux.db/id :stock/Pu
:commod :commodity/Pu
:weight-ton 23 }
#inst "2115-02-14T18"]
[:crux.tx/put
{:crux.db/id :stock/Pu
:commod :commodity/Pu
:weight-ton 22.2 }
#inst "2115-02-15T18"]
[:crux.tx/put
{:crux.db/id :stock/Pu
:commod :commodity/Pu
:weight-ton 24 }
#inst "2115-02-18T18"]
[:crux.tx/put
{:crux.db/id :stock/Pu
:commod :commodity/Pu
:weight-ton 24.9 }
#inst "2115-02-19T18"]])
;;=> #:crux.tx{:tx-id 1, :tx-time #inst "2020-06-18T14:14:08.347-00:00"}(crux/submit-tx node
[[:crux.tx/put
{:crux.db/id :stock/N
:commod :commodity/N
:weight-ton 3 }
#inst "2115-02-13T18" ;; start valid-time
#inst "2115-02-19T18"] ;; end valid-time
[:crux.tx/put
{:crux.db/id :stock/CH4
:commod :commodity/CH4
:weight-ton 92 }
#inst "2115-02-15T18"
#inst "2115-02-19T18"]])
;;=> #:crux.tx{:tx-id 2, :tx-time #inst "2020-06-18T14:15:19.716-00:00"}You notice that the amount of Nitrogen and Methane has not changed in the last week which saves you some time
(crux/entity (crux/db node #inst "2115-02-14") :stock/Pu)
;;=> {:crux.db/id :stock/Pu, :commod :commodity/Pu, :weight-ton 21}
(crux/entity (crux/db node #inst "2115-02-18") :stock/Pu)
;;=> {:crux.db/id :stock/Pu, :commod :commodity/Pu, :weight-ton 22.2}The CEO is impressed with your speed, but a little skeptical that you have done it properly.
You gain their confidence by showing them the entries for Plutonium on two different days:
(defn easy-ingest
"Uses Crux put transaction to add a vector of documents to a node"
[node docs]
(crux/submit-tx node (mapv (fn [doc] [:crux.tx/put doc]) docs)))As a parting gift to them you create an easy ingest function so that if they needed to add more commodities to their stock list they could do it fast.
Tombaugh Resources Ltd. are happy that this will be simple enough to use. They thank you for the extra help and you head back to your ship.
Congratulations on completing your first assignment.
We would like you to go to Mercury, the hub of the trade world. Their main trade center has a new IT department and want you to show them how to query Crux.
— Helios Banking Inc.
You are back at your ship and check your communications panel. There is a new assignment waiting for you:
(crux/submit-tx
node
[[:crux.tx/put
(assoc manifest :badges ["SETUP" "PUT"])]])
;;=> #:crux.tx{:tx-id 3, :tx-time #inst "2020-06-18T14:20:31.602-00:00"}It’s along flight so you refuel, and update your manifest.
You have been awarded a new badge.
The Clojure function assoc to save time.
You come into range of the satellites orbiting Mecury.
Your communications panel lights up.
The government is asking to see your flight manifest.
You have permission to land
Greetings.
You have reached Mercury, home to the greatest stock market in the Solar System. Your ship has been flagged as a transport vessel for business related activities.
Please have your flight manifest ready and prepare to land.
— Mercury Commonwealth
A Datalog query consists of a set of variables and a set of clauses. The result of running a query is a result set (or lazy sequence) of the possible combinations of values that satisfy all of the clauses at the same time. These combinations of values are referred to as "tuples".
The possible values within the result tuples are derived from your database of documents.
The documents themselves are represented in the database indexes as "entity–attribute–value" (EAV) facts. For example, a single document
{:crux.db/id :myid :color "blue" :age 12}
is transformed into two facts
[[:myid :color "blue"][:myid :age 12]]
In the most basic case, a Datalog query works by searching for "subgraphs" in the database that match the pattern defined by the clauses. The values within these subgraphs are then returned according to the list of return variables requested in the :find vector within the query.
You land on the surface of the tidally locked planet. As you do, the job ticket for this assignment is issued.
| Task |
Find information on products for stock buyers |
| Company |
Interplanetary Buyers & Sellers (IPBS) |
| Contact |
Cosmina Sinnett |
| Submitted |
2115-06-20T10:54:27 |
| Additional information |
We have some new starters in the sales team. They need to be trained on how to query Crux using Datalog to quickly find the information they need on a product. Traders must have access to up-to-date information when talking to their clients. We would also like you to create a function that can be used for the things we have to look up a lot. I will include example data so they can learn using relevant commodities. |
| Attachments |
Attachments :: example_data.txt |
On your way over to the IPBS office you input the data in the attachment using the easy ingest function you created on Pluto. This means you are ready to give them a tutorial when you get there.
Oh good, you’re here. I have a room reserved and we have five new starters ready and waiting to learn how to query Crux.
We are in the middle of our double sunrise. The workers take this time to rest, but in half an Earth hour the sun will rise again and the workers will start back.
You can take this time to prepare any training material if you wish.
— Cosmina Sinnett, Head of Training - IPBS
You have the opportunity to prepare examples for the lesson ahead.
(crux/q (crux/db node)
'{:find [element]
:where [[element :type :element/metal]]})
;;=> #{[:commodity/Pu] [:commodity/Au]}This basic query is returning all the elements that are defined as :element/metal. The :find clause tells Crux what variables you want to return.
In this case we are returning the :crux.db/id due to our placement of element.
(=
(crux/q (crux/db node)
'{:find [element]
:where [[element :type :element/metal]]})
(crux/q (crux/db node)
{:find '[element]
:where '[[element :type :element/metal]]})
(crux/q (crux/db node)
(quote
{:find [element]
:where [[element :type :element/metal]]})))
;;=> trueThe vectors given to the clauses should be quoted. How you do it at this stage is arbitrary, but it becomes more important if you are using :args, which we will cover momentarily.
(crux/q (crux/db node)
'{:find [name]
:where [[e :type :element/metal]
[e :common-name name]]})
;;=> #{["Gold"] ["Plutonium"]}To find all the names of the commodities that have a certain property, such as :type, you need to use a combination of clauses. Here we have bound the results of type :element/metal to e. Next, we can use the results bound to e and bind the :common-name of them to name. name is what has been specified to be returned and so our result is the common names of all the elements that are metals.
One way to think of this is that you are filtering to only get the results that satisfy all the clauses.
(crux/q (crux/db node)
'{:find [name rho]
:where [[e :density rho]
[e :common-name name]]})
;;=> #{["Nitrogen" 1.2506] ["Carbon" 2.267] ["Methane" 0.717] ["Borax" 1.73] ["Gold" 19.3] ["Plutonium" 19.816]}You can pull out as much data as you want into your result tuples by adding additional variables to the :find clause.
The example above returns the :density and the :common-name values for all entities in Crux that have values of some kind for both :density and :common-name attributes.
(crux/q (crux/db node)
{:find '[name]
:where '[[e :type t]
[e :common-name name]]
:args [{'t :element/metal}]})
;;=> #{["Gold"] ["Plutonium"]}:args can be used to further filter the results. Lets break down what is going down here.
First, we are assigning all :crux.db/id that have a :type to e:
(crux/q (crux/db node)
{:find '[name]
:where '[[e :type t]
[e :common-name name]]
:args [{'t :element/metal}]})
;;=> #{["Gold"] ["Plutonium"]}
e
|
← |
At the same time we are assigning all the :types to t:
(crux/q (crux/db node)
{:find '[name]
:where '[[e :type t]
[e :common-name name]]
:args [{'t :element/metal}]})
;;=> #{["Gold"] ["Plutonium"]}
t
|
← |
Then we assign all the names within e that have a :common-name to name:
(crux/q (crux/db node)
{:find '[name]
:where '[[e :type t]
[e :common-name name]]
:args [{'t :element/metal}]})
;;=> #{["Gold"] ["Plutonium"]}
name
|
← |
(crux/q (crux/db node)
{:find '[name]
:where '[[e :type t]
[e :common-name name]]
:args [{'t :element/metal}]})
;;=> #{["Gold"] ["Plutonium"]}We have specified that we want to get the names out, but not before looking at :args
In :args we have further filtered the results to only show us the names of that have :type :element/metal.
We could have done that inside the :where clause, but using :args removes the need for hard-coding inside the query clauses.
You give your lesson to the new starters when they return. They are a good audience and follow it well.
To check their understanding you set them a task to create a function to aid their daily queries. You are impressed with their efforts.
When you are finished, Cosmina thanks you and you head back to the space port.
(defn filter-type
[type]
(crux/q (crux/db node)
{:find '[name]
:where '[[e :type t]
[e :common-name name]]
:args [{'t type}]}))
(defn filter-appearance
[description]
(crux/q (crux/db node)
{:find '[name IUPAC]
:where '[[e :common-name name]
[e :IUPAC-name IUPAC]
[e :appearance appearance]]
:args [{'appearance description}]}))
(filter-type :element/metal)
;;=> #{["Gold"] ["Plutonium"]}
(filter-appearance "white solid")
;;=> #{["Borax" "Sodium tetraborate decahydrate"]}You are back at your spaceship. Seeing another light on your communications panel, you realize there is another assignment ready for you.
Congratulations on completing your assignment. You are getting the hang of things now, and we are impressed with your progress.
We would like you to go to Neptune. They have recently lost a lot of data in a flood so they have decided to digitize their entire system and archives. We told them you could do it in such a way that the data is still time ordered as it was with their previous filing system.
Good luck, and don’t forget to update your manifest.
— Banking Inc.
You update your manifest with the latest badge.
(crux/submit-tx
node [[:crux.tx/put (assoc manifest
:badges ["SETUP" "PUT" "DATALOG-QUERIES"])]])
;;=> #:crux.tx{:tx-id 1, :tx-time #inst "2020-06-18T14:31:46.148-00:00"}
You enter the Neptunian atmosphere and your communications panel lights up.
It is our honor to welcome you to the planet Neptune.
If you are visiting for business reasons, please present your manifest.
Otherwise, have your visa ready for inspection.
— Poseidon Republic of Wealth
The government is asking to see your flight manifest.
You have permission to land
One or more documents can be inserted into Crux via a put transaction at a specific valid-time. The valid-time can be any time (past, future or present).
If no valid-time is provided, Crux will default to the transaction time, i.e. the present. Each document survives until it is deleted or a new version of it is added.
Upon landing on the ice giant, your communications panel lights up indicating that the job ticket is available.
| Task |
Back fill insurance documents |
| Company |
Coast Insurance |
| Contact |
Lyndon Mercia-York |
| Submitted |
2115-02-22T13:38:20 |
| Additional information |
We have lost a lot of our records in a flood. I think it is prudent to start storing our data digitally. It is important to track policy holders' level of cover at the time of the incident. We have read a lot about the conveniences of Crux’s bitemporality in this situation. We need you to help us get up and running. Please send someone quickly though - the waters are rising. |
| Attachments |
— |
Thank goodness you’re here.
We need you to show us how to put our customers information into Crux in order of time. Working with insurance claims, we should be able to easily look back in time at what type of coverage the customer had at the time of the incident.
Are you able to help us?
— Lyndon Mercia-York
Outside your ship you are met by a panicked looking Lyndon.
Lyndon gives you some data for a client that you can use as an example. Coast Insurance need to know what kind of cover each customer has and if it was valid at a given time.
You show them how to ingest a document using a valid-time so that the information is backdated to when the customer took the cover out.
(crux/submit-tx
node
[[:crux.tx/put
{:crux.db/id :consumer/RJ29sUU
:consumer-id :RJ29sUU
:first-name "Jay"
:last-name "Rose"
:cover? true
:cover-type :Full}
#inst "2114-12-03"]])
;;=> #:crux.tx{:tx-id 0, :tx-time #inst "2020-06-18T14:38:29.240-00:00"}The company needs to know the history of insurance for each cover. You show them how to use the bitemporality of Crux to do this.
(crux/submit-tx
node
[[:crux.tx/put (1)
{:crux.db/id :consumer/RJ29sUU
:consumer-id :RJ29sUU
:first-name "Jay"
:last-name "Rose"
:cover? true
:cover-type :Full}
#inst "2113-12-03" ;; Valid time start
#inst "2114-12-03"] ;; Valid time end
[:crux.tx/put (2)
{:crux.db/id :consumer/RJ29sUU
:consumer-id :RJ29sUU
:first-name "Jay"
:last-name "Rose"
:cover? true
:cover-type :Full}
#inst "2112-12-03"
#inst "2113-12-03"]
[:crux.tx/put (3)
{:crux.db/id :consumer/RJ29sUU
:consumer-id :RJ29sUU
:first-name "Jay"
:last-name "Rose"
:cover? false}
#inst "2112-06-03"
#inst "2112-12-02"]
[:crux.tx/put (4)
{:crux.db/id :consumer/RJ29sUU
:consumer-id :RJ29sUU
:first-name "Jay"
:last-name "Rose"
:cover? true
:cover-type :Promotional}
#inst "2111-06-03"
#inst "2112-06-03"]])
;;=> #:crux.tx{:tx-id 1, :tx-time #inst "2020-06-18T14:39:11.189-00:00"}-
This is the insurance that the customer had last year. Along with the start valid-time you use an end valid-time so as not to affect the most recent version of the document.
-
This is the previous insurance plan. Again, you use a start and end valid-time.
-
There was a period when the customer was not covered,
-
and before that the customer was on a promotional plan.
You now show them a few queries. You know that you can query Crux as of a given valid-time. This shows the state of Crux at that time.
First you chose a date that the customer had full cover:
(crux/q (crux/db node #inst "2115-07-03")
'{:find [cover type]
:where [[e :consumer-id :RJ29sUU]
[e :cover? cover]
[e :cover-type type]]})
;;=> #{[true :Full]}Next you show them a query for a the customer in a time when they had a different type of cover:
(crux/q (crux/db node #inst "2111-07-03")
'{:find [cover type]
:where [[e :consumer-id :RJ29sUU]
[e :cover? cover]
[e :cover-type type]]})
;;=> #{[true :Promotional]}And finally you show them a time when the customer had no cover at all.
(crux/q (crux/db node #inst "2112-07-03")
'{:find [cover type]
:where [[e :consumer-id :RJ29sUU]
[e :cover? cover]
[e :cover-type type]]})
;;=> #{}Confident in their ability to put the remainder of their records into Crux, Lyndon thanks you.
You say goodbye to Lyndon and head back to the space port.
I can’t believe we’ve not digitized sooner. There was a huge push to start using more paper as the Neptune tree population was getting out of control from the accelerated terraforming, but since all these floods I’m not sure paper was the right choice.
— Lyndon Mercia-York
Back at your spaceship you check your communications panel. There is a new assignment waiting for you.
We have assigned you a quick task on Saturn helping a small company who are having some problems keeping their records in order.
This shouldn’t take long, but don’t forget they will still need to see your manifest.
— Helios Banking Inc.
You add the new badge to your manifest
(crux/submit-tx
node [[:crux.tx/put
(assoc manifest
:badges ["SETUP" "PUT" "DATALOG-QUERIES" "BITEMP"])]])
;;=> #:crux.tx{:tx-id 2, :tx-time #inst "2020-06-18T14:46:47.606-00:00"}
As you pass through the innermost ring of Saturn, A warning light appears on your communications panel. You open the message. It’s from Space Customs and reads:
They are asking to see your flight manifest.
You have permission to land
We extend you the warmest welcome.
We must check papers before we can give you permission to land.
— Cronus Peaceful Nations
| Transaction | Description |
|---|---|
|
Writes a version of a document |
|
Deletes a version of a document |
|
Stops a transaction if the precondition is not met. |
|
Removes a document entirely |
match checks the current state of an entity - if the entity doesn’t match the provided doc, the transaction will not continue.
You can also pass nil to check that the entity doesn’t exist prior to your transaction.
A match transaction takes the entity id, along with an expected document. Optionally you can provide a valid time.
- Time in Crux is denoted
-
#inst "yyyy-MM-ddThh:mm:SS" - For example, 9:30 pm on January 2nd 1999 would be written:
-
#inst "1999-01-02T21:30:00". - A complete match transaction has the form
-
[:crux.tx/match entity-id expected-doc valid-time]
| Task |
Secure trading system |
| Company |
Cronus Market Technologies |
| Contact |
Ubuku Eppimami |
| Submitted |
2115-02-23T13:38:20 |
| Additional information |
We need to be shown how to ensure no trades are done without the buyer and seller having the necessary funds or stock respectively |
| Attachments |
The next shuttle to the CMT office leaves in 5 Earth minutes. While you wait you use your easy ingest function you created on Pluto to put the example data into your system. You also decide to make some Clojure functions so you can easily show Ubuku the stock and fund levels after the trades.
Just as you are finishing off your shuttle arrives.
After a short journey through the icy lower clouds of Saturn you are met by a friendly faced Ubuku.
(defn stock-check
[company-id item]
{:result (crux/q (crux/db node)
{:find '[name funds stock]
:where ['[e :company-name name]
'[e :credits funds]
['e item 'stock]]
:args [{'e company-id}]})
:item item})
(defn format-stock-check
[{:keys [result item] :as stock-check}]
(for [[name funds commod] result]
(str "Name: " name ", Funds: " funds ", " item " " commod)))You explain to Ubuku that all they need to do to solve their problem is to use the match operation along with put when they are processing their trades.
Hello friend.
We have been using Crux for a short time now and think it is great. The problem is a human one. Occasionally we process trades without checking that are enough funds in the buyers account.
I know there is a way that we can stop this happening in Crux.
I sent you some example data in the job ticket for you to use, did you find it alright?
— Ubuku Eppimami
You show Ubuku the match operation for a valid transaction. You move 10 units of Methane (:units/CH4) each at the cost of 100 credits to Blue Energy:
You explain that because the provided doc is as expected for both the buyer and the seller that the transaction goes through.
(crux/submit-tx
node
[[:crux.tx/match
:blue-energy
{:crux.db/id :blue-energy
:seller? false
:buyer? true
:company-name "Blue Energy"
:credits 1000}]
[:crux.tx/put
{:crux.db/id :blue-energy
:seller? false
:buyer? true
:company-name "Blue Energy"
:credits 900
:units/CH4 10}]
[:crux.tx/match
:tombaugh-resources
{:crux.db/id :tombaugh-resources
:company-name "Tombaugh Resources Ltd."
:seller? true
:buyer? false
:units/Pu 50
:units/N 3
:units/CH4 92
:credits 51}]
[:crux.tx/put
{:crux.db/id :tombaugh-resources
:company-name "Tombaugh Resources Ltd."
:seller? true
:buyer? false
:units/Pu 50
:units/N 3
:units/CH4 82
:credits 151}]])
;;=> #:crux.tx{:tx-id 0, :tx-time #inst "2020-06-18T15:37:20.271-00:00"}You show Ubuku the result of the trade using the function you created earlier:
They are happy that this works as he sees the 1000 credits move from Blue energy to Tombaugh Resources Ltd. and 10 units of Methane the other way.
Ubuku asks if you can show them what would happen if there was not enough funds in the account of a buyer.
(format-stock-check (stock-check :tombaugh-resources :units/CH4))
;;=> ("Name: Tombaugh Resources Ltd., Funds: 151, :units/CH4 82")(format-stock-check (stock-check :blue-energy :units/CH4))
;;=> ("Name: Blue Energy, Funds: 900, :units/CH4 10")You show him a trade where the old doc is not as expected for Encompass trade, to buy 10,000 units of Gold from Gold Harmony.
You explain to Ubuku that this time, because you have both match operations in the same transaction, the trade does not go through.
The accounts remain the same, even though the failing match was the second operation.
Ubuku thanks you. This is just what they are looking for. You head back to the space station to see if there is another assignment waiting for you.
(crux/submit-tx
node
[[:crux.tx/match
:gold-harmony
{:crux.db/id :gold-harmony
:company-name "Gold Harmony"
:seller? true
:buyer? false
:units/Au 10211
:credits 51}]
[:crux.tx/put
{:crux.db/id :gold-harmony
:company-name "Gold Harmony"
:seller? true
:buyer? false
:units/Au 211
:credits 51}]
[:crux.tx/match
:encompass-trade
{:crux.db/id :encompass-trade
:company-name "Encompass Trade"
:seller? true
:buyer? true
:units/Au 10
:units/Pu 5
:units/CH4 211
:credits 100002}]
[:crux.tx/put
{:crux.db/id :encompass-trade
:company-name "Encompass Trade"
:seller? true
:buyer? true
:units/Au 10010
:units/Pu 5
:units/CH4 211
:credits 1002}]])
;;=> #:crux.tx{:tx-id 1, :tx-time #inst "2020-06-18T15:23:38.540-00:00"}
(format-stock-check (stock-check :gold-harmony :units/Au))
;;=> ("Name: Gold Harmony, Funds: 51, :units/Au 10211")
(format-stock-check (stock-check :encompass-trade :units/Au))
;;=> ("Name: Encompass Trade, Funds: 1002, :units/Au 10")Back at the spaceship there is a light waiting for you on your communications panel.
Well done, you’ve had a productive week. We have one final task for you to do before you finish for the week
You need to go to Jupiter and meet Kaarlang, it’s his last day working for us and he needs to delete his trade clients from his personal Crux node for data protection.
— Helios Banking Inc.
You update your manifest with your most recent badge.
As you do so, you check to see if you still have the note that the porter gave you for Kaarlang back on Earth.
(crux/submit-tx
node [[:crux.tx/put
(assoc manifest
:badges ["SETUP" "PUT" "DATALOG-QUERIES" "BITEMP" "MATCH"])]])
;;=> #:crux.tx{:tx-id 3, :tx-time #inst "2020-06-18T15:24:39.037-00:00"}(crux/q (crux/db node)
{:find '[belongings]
:where '[[e :cargo belongings]]
:args [{'belongings "secret note"}]})
;;=> #{["secret note"]}
You approach Jupiter and marvel at its bands. You wish you could have seen it this close before the Great Red Spot dissipated.
As you enter the Jovian atmosphere your communications panel lights up with the now expected, but rather terse message from boundary control.
The government is asking to see your flight manifest.
You have your manifest ready and so have permission to land
Jupiter’s boundary is controlled.
If you wish to enter, present your papers now.
— Zeus Confederacy
It’s a turbulent ride down to the space port.
To take your mind off the colossal storm outside, you check the Crux manual for the delete operation.
| Operation | Description |
|---|---|
|
Writes a version of a document |
|
Deletes a version of a document |
|
Stops a transaction if the precondition is not met. |
|
Removes a document entirely |
Delete:
The delete operation takes a valid eid with the option to include a start and end valid-time.
- Time in Crux is denoted
-
#inst "yyyy-MM-ddThh:mm:SS" - For example, 9:30 pm on January 2nd 1999 would be written:
-
#inst "1999-01-02T21:30:00".
The document will be deleted as of the transaction time, or between the start and end valid-time if provided.
Historical versions of the document that fall outside of the valid-time window will be preserved.
- A complete delete transaction has the form
-
[:crux.tx/delete eid valid-time-start valid-time-end]
| Task |
|
| Company |
|
| Contact |
|
| Submitted |
|
| Additional information |
|
| Attachments |
— |
As you leave your ship, you are met by the martian Kaarlang:
Yes, we’ll work together to do this.
Kaarlang gives you his client history so you can sync up your Crux node.
Hi there, I believe you’re here to help me.
I’ve been told I need to delete my client history with today being my last day.
Is this something you can help me with?
You input the client history
(crux/submit-tx
node [[:crux.tx/put {:crux.db/id :kaarlang/clients
:clients [:encompass-trade]}
#inst "2110-01-01T09"
#inst "2111-01-01T09"]
[:crux.tx/put {:crux.db/id :kaarlang/clients
:clients [:encompass-trade :blue-energy]}
#inst "2111-01-01T09"
#inst "2113-01-01T09"]
[:crux.tx/put {:crux.db/id :kaarlang/clients
:clients [:blue-energy]}
#inst "2113-01-01T09"
#inst "2114-01-01T09"]
[:crux.tx/put {:crux.db/id :kaarlang/clients
:clients [:blue-energy :gold-harmony :tombaugh-resources]}
#inst "2114-01-01T09"
#inst "2115-01-01T09"]])
;;=> #:crux.tx{:tx-id 0, :tx-time #inst "2020-06-18T15:55:00.894-00:00"}To get a good visual aid, you show Kaarlang how to view his client history.
This way you both can see when the clients are deleted.
You use the option :with-docs? to show the content of the doc.
The result shows the names of the clients that have been assigned to Kaarlang since he started at the company in 2110.
(crux/entity-history
(crux/db node #inst "2116-01-01T09")
:kaarlang/clients
:desc
{:with-docs? true})
#_#_
=> [{:crux.tx/tx-time #inst "2020-06-18T15:57:28.900-00:00",
:crux.tx/tx-id 1,
:crux.db/valid-time #inst "2114-01-01T09:00:00.000-00:00",
:crux.db/content-hash #crux/id "d4bca6c78409d9d40ee42319a8aec32bffad9030",
:crux.db/doc
{:crux.db/id :kaarlang/clients,
:clients [:blue-energy :gold-harmony :tombaugh-resources]}}
{:crux.tx/tx-time #inst "2020-06-18T15:57:28.900-00:00",
:crux.tx/tx-id 1,
:crux.db/valid-time #inst "2113-01-01T09:00:00.000-00:00",
:crux.db/content-hash #crux/id "000e5b775b55d06f0bddc77d736184284aa1e4e9",
:crux.db/doc {:crux.db/id :kaarlang/clients, :clients [:blue-energy]}}
{:crux.tx/tx-time #inst "2020-06-18T15:57:28.900-00:00",
:crux.tx/tx-id 1,
:crux.db/valid-time #inst "2111-01-01T09:00:00.000-00:00",
:crux.db/content-hash #crux/id "cd71551fe21219db59067ce7483370fdebaae8b0",
:crux.db/doc
{:crux.db/id :kaarlang/clients, :clients [:encompass-trade :blue-energy]}}
{:crux.tx/tx-time #inst "2020-06-18T15:57:28.900-00:00",
:crux.tx/tx-id 1,
:crux.db/valid-time #inst "2110-01-01T09:00:00.000-00:00",
:crux.db/content-hash #crux/id "5ec42ea653288e01e1a9d7d2068b4658416177e0",
:crux.db/doc {:crux.db/id :kaarlang/clients, :clients [:encompass-trade]}}]Next you delete the whole history of clients buy choosing a start and end valid-time that spans his entire employment time.
(crux/submit-tx
node [[:crux.tx/delete :kaarlang/clients #inst "2110-01-01" #inst "2116-01-01"]])
;;=> #:crux.tx{:tx-id 1, :tx-time #inst "2020-06-18T15:59:38.323-00:00"Using the same method as before you show Kaarlang the effect of this operation.
Kaarlang is impressed it is that easy. You point out that there are no longer any documents attached to the transactions.
(crux/entity-history
(crux/db node #inst "2116-01-01T09")
:kaarlang/clients
:desc
{:with-docs? true})
#_#_
=> [{:crux.tx/tx-time #inst "2020-06-18T15:59:38.323-00:00",
:crux.tx/tx-id 2,
:crux.db/valid-time #inst "2115-01-01T09:00:00.000-00:00",
:crux.db/content-hash #crux/id "0000000000000000000000000000000000000000",
:crux.db/doc nil}
{:crux.tx/tx-time #inst "2020-06-18T15:59:38.323-00:00",
:crux.tx/tx-id 2,
:crux.db/valid-time #inst "2114-01-01T09:00:00.000-00:00",
:crux.db/content-hash #crux/id "0000000000000000000000000000000000000000",
:crux.db/doc nil}
{:crux.tx/tx-time #inst "2020-06-18T15:59:38.323-00:00",
:crux.tx/tx-id 2,
:crux.db/valid-time #inst "2113-01-01T09:00:00.000-00:00",
:crux.db/content-hash #crux/id "0000000000000000000000000000000000000000",
:crux.db/doc nil}
{:crux.tx/tx-time #inst "2020-06-18T15:59:38.323-00:00",
:crux.tx/tx-id 2,
:crux.db/valid-time #inst "2111-01-01T09:00:00.000-00:00",
:crux.db/content-hash #crux/id "0000000000000000000000000000000000000000",
:crux.db/doc nil}
{:crux.tx/tx-time #inst "2020-06-18T15:59:38.323-00:00",
:crux.tx/tx-id 2,
:crux.db/valid-time #inst "2110-01-01T09:00:00.000-00:00",
:crux.db/content-hash #crux/id "0000000000000000000000000000000000000000",
:crux.db/doc nil}
{:crux.tx/tx-time #inst "2020-06-18T15:59:38.323-00:00",
:crux.tx/tx-id 2,
:crux.db/valid-time #inst "2110-01-01T00:00:00.000-00:00",
:crux.db/content-hash #crux/id "0000000000000000000000000000000000000000",
:crux.db/doc nil}]You remember the secret note in your pocket and pass it to Kaarlang.
I am grateful that you took the time to show me this.
Today is a sad day for me as I have very much enjoyed my time here.
Although, I was expecting to hear from a friend before I left. There is a very important message that I am waiting for.
Kaarlang reads the note.
He looks at you with the a peculiar facial expression.
This is the note I was waiting for.
It has information about the a top secret stellar transport shuttle.
If you are interested in a great adventure, the shuttle comes once every hundred years or so to take a select few to a nearby star system. The system is home to a mysterious hyper-intelligent form of life.
I’m sure there would be a place for you if you were willing to help out. The passengers on the shuttle have the right to be forgotten. We need someone that can remove the passengers data from the solar system.
What do you think?
You arrive at the comet 'Oumuamua and pull along side, asking for permission to land. A voice comes over the communications system
How did you find us? Who sent you?? — Mysterious person
"Kaarlang sent me" You have permission to land
You land on the space port and are ushered inside. The ships captain, Ilex, greets you.
You are excited by the prospect and agree to help. First you read the manual entry for evict as this will be the perfect tool.
Hello, it’s good to have you with us.
We are set to leave the solar system right away and as part of our service we offer people the right to be forgotten. Some are not worried that their information is kept here, however others want there to be no personal data left behind.
You may not have been told this yet, but this comet is actually a transportation vessel. It will take us to the star system Gilese 667C which is home to intelligent life far superior to our own. We all are hoping to find opportunities beyond our wildest dreams. All records of this transportation vessel and any life outside of the solar system are heavily monitored and wiped in the interest of preserving the normal technological advancement of the Human race. This means we know little of the beings we are going to meet.
Our task for you is to remove the records of the people who have chosen to be forgotten here.
— Ilex, Captain
You are given the data for the people on the ship and sync up your Crux node. You decide that you are going to embark on this adventure along with them so you add your name to the list.
| Operation | Description |
|---|---|
|
Writes a version of a document |
|
Deletes a version of a document |
|
Stops a transaction if the precondition is not met. |
|
Removes a document entirely |
Evict:
Crux supports eviction of active and historical data to assist with technical compliance for information privacy regulations.
The main transaction log contains only hashes and is immutable. All document content is stored in a dedicated document log that can be evicted by compaction.
- A complete evict transaction has the form
-
[:crux.tx/evict eid]
(crux/submit-tx node
[[:crux.tx/put
{:crux.db/id :person/kaarlang
:full-name "Kaarlang"
:origin-planet "Mars"
:identity-tag :KA01299242093
:DOB #inst "2040-11-23"}]
[:crux.tx/put
{:crux.db/id :person/ilex
:full-name "Ilex Jefferson"
:origin-planet "Venus"
:identity-tag :IJ01222212454
:DOB #inst "2061-02-17"}]
[:crux.tx/put
{:crux.db/id :person/thadd
:full-name "Thad Christover"
:origin-moon "Titan"
:identity-tag :IJ01222212454
:DOB #inst "2101-01-01"}]
[:crux.tx/put
{:crux.db/id :person/johanna
:full-name "Johanna"
:origin-planet "Earth"
:identity-tag :JA012992129120
:DOB #inst "2090-12-07"}]])
;;=> #:crux.tx{:tx-id 0, :tx-time #inst "2020-06-18T16:11:03.410-00:00"}Before you start the eviction process you make a query function so you can see the full results of anything stored in Crux:
(defn full-query
[node]
(crux/q
(crux/db node)
'{:find [id]
:where [[e :crux.db/id id]]
:full-results? true}))You show the others the result:
(full-query node)
#_#_
=> #{[{:crux.db/id :person/ilex,
:full-name "Ilex Jefferson",
:origin-planet "Venus",
:identity-tag :IJ01222212454,
:DOB #inst "2061-02-17T00:00:00.000-00:00"}]
[{:crux.db/id :person/thadd,
:full-name "Thad Christover",
:origin-moon "Titan",
:identity-tag :IJ01222212454,
:DOB #inst "2101-01-01T00:00:00.000-00:00"}]
[{:crux.db/id :person/kaarlang,
:full-name "Kaarlang",
:origin-planet "Mars",
:identity-tag :KA01299242093,
:DOB #inst "2040-11-23T00:00:00.000-00:00"}]
[{:crux.db/id :person/johanna,
:full-name "Johanna",
:origin-planet "Earth",
:identity-tag :JA012992129120,
:DOB #inst "2090-12-07T00:00:00.000-00:00"}]}The Crux manual said that the evict operation will remove a document entirely. Ilex tells you the only person who whishes to exercise their right to be forgotten is Kaarlang.
(crux/submit-tx node [[:crux.tx/evict :person/kaarlang]])
;;=> #:crux.tx{:tx-id 3, :tx-time #inst "2020-06-18T16:13:03.276-00:00"}You use your function and see that the transaction was a success.
All the data associated with the specified :crux.db/id has been removed from the Crux along with the eid itself.
(full-query node)
#_#_
=> #{[{:crux.db/id :person/ilex,
:full-name "Ilex Jefferson",
:origin-planet "Venus",
:identity-tag :IJ01222212454,
:DOB #inst "2061-02-17T00:00:00.000-00:00"}]
[{:crux.db/id :person/thadd,
:full-name "Thad Christover",
:origin-moon "Titan",
:identity-tag :IJ01222212454,
:DOB #inst "2101-01-01T00:00:00.000-00:00"}]
[{:crux.db/id :person/johanna,
:full-name "Johanna",
:origin-planet "Earth",
:identity-tag :JA012992129120,
:DOB #inst "2090-12-07T00:00:00.000-00:00"}]}This means the transactions will never be removed. You assure Ilex that the documents are completely removed from Crux, you can show this by looking at the history.
You show the results to Kaarlang who is happy that there his details are no longer a part of the ships logs.
The transaction history is immutable.
(crux/entity-history
(crux/db node)
:person/kaarlang
:desc
{:with-docs? true})
#_#_
=> [{:crux.tx/tx-time #inst "2020-06-18T16:11:03.410-00:00",
:crux.tx/tx-id 2,
:crux.db/valid-time #inst "2020-06-18T16:11:03.410-00:00",
:crux.db/content-hash #crux/id "c3ad3191fff06083fedf3640b625566c02033a6b",
:crux.db/doc
#:crux.db{:id #crux/id "efe634523d6867a3c6e4089074adf29b07b45f43",
:evicted? true}}]
Ilex thanks you and takes you to the Cryogenics department. You must be put into stasis as the journey will take around 25 years, even at the near light speeds of the ship.
You are astonished with the amount that you have done in one short week. How did little old you end up with an opportunity as big as this?
Your eyes get heavy as the cryogenicist initiates the hibernation process. As they do, you wonder if you’ll ever come back to the solar system.