OT Editor - Documentation #340
Open
+273
−4
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
angary
reviewed
Mar 15, 2023
angary
reviewed
Mar 15, 2023
😎 Co-authored-by: Gary Sun <[email protected]>
angary
reviewed
Mar 18, 2023
angary
approved these changes
Mar 19, 2023
Co-authored-by: Gary Sun <[email protected]>
Co-authored-by: Gary Sun <[email protected]>
|
lgtm, main thing is i noticed in a lot of places the docs say "it's complicated" or "it's complex" - these are fluff sentences that imo not only don't add to documentation/explanation but make it harder for people reading to engage. will provide more detailed review as well |
hanyuone
requested changes
Jun 3, 2023
| @@ -0,0 +1,163 @@ | |||
| # The Concurrent Editor | |||
| As a high level overview though the concurrent editor is an OT based editor thats heavily influenced by Google's Wave algorithm, Wave was Google's predecessor to Google docs and heavily inspired the architecture for Google Docs. Our editor uses the same underlying architecture as Wave except the difference lies in the type of operations we are sending between the server and client. Wave was designed to operate with operations that modified unstructured data, while our operations modify structured JSON data. | |||
There was a problem hiding this comment.
"As a high level overview though, the concurrent editor"
| ## High Level Architecture | ||
| At a very high level our editor server consists of 3 distinct layers. | ||
| - **The data layer** | ||
| - This layer is the server's copy of the current state of the document. This layer is what all incoming operations will modify. Our initial design for the data layer involved a singular struct modelling the entire document that we modified using reflection. This proved tricky due to the intricacies of Go's reflection system so we moved to an AST based approach. Currently the data layer is just the AST for the JSON of the document, and operations modify this AST directly. To prevent corrupted documents we have various data integrity checks utilising reflection. |
There was a problem hiding this comment.
is it necessary to mention reflection (i.e. is it still in the codebase)? if you really want to include it, it could be in some other document/section, but not here if the old approach is completely removed
| I personally feel like its rather easy to understand a system if you understand how it achieves its key features. This section is about what exactly happens when a user clicks the "edit" button on a document and constructs an edit session. What type of objects are created? Where do they live? Etc. | ||
|
|
||
| ### The connection starts | ||
| When the user clicks the "edit" button, this instantiates a HTTP request that's handled by the HTTP handler in `main.go`: `func EditEndpoint(w http.ResponseWriter, r *http.Request)`. This handler takes the incoming request, looks up the requested document and if it exists upgrades the connection to a WebSocket connection. This is important as a WebSocket connection allows for bidirectional communication between the client and server in real time without needing either to poll for updates |
| Remove | ||
| ) | ||
| ``` | ||
| The above code snippet uniquely defines an operation. Operations take a path to where in the document AST they are being applied (see the paper on tree based transform functions) and the physical operation being applied. The operation being applied (`OperationModel`) is actually an interface and is the reason why parsing operations is more complex than it seems. This interface defines two functions, one for transforming a operation against another operation and one for applying an operation to an AST. |
There was a problem hiding this comment.
would be nice to link the papers in the docs themselves, saves anyone reading having to click back and forth
| ``` | ||
| The above code snippet uniquely defines an operation. Operations take a path to where in the document AST they are being applied (see the paper on tree based transform functions) and the physical operation being applied. The operation being applied (`OperationModel`) is actually an interface and is the reason why parsing operations is more complex than it seems. This interface defines two functions, one for transforming a operation against another operation and one for applying an operation to an AST. | ||
|
|
||
| The reason why `OperationModel` is an interface is because there are several distinct types of operations we can apply for varying types. Theres different operations for editing an `integer`, `array`, `boolean`, `object` and `string` field. Each of these define their own transformation functions and application logic, so in order to maintain a clean abstraction we use an interface. As an example this is how the `string` operation type implements this interface , its a rather intense implementation since it also implements string based transform functions. |
There was a problem hiding this comment.
can get rid of the "it's a rather intense implementation" phrase
| }, | ||
| } | ||
| ``` | ||
| the configuration is in essence a mapping between the the `reflect.Type` representation of the interface and the `reflect.type` representation of every struct that "implements it". Implements is in quote as there is no way to statically verify this at compile time, instead if any of these config options are invalid a runtime error will be thrown during parsing. Usage of the `cmsjson` library for the most part is rather simple (thanks to generics in Go :D). An example can be found in the `ParseOperation` function within `operation_model.go`. |
There was a problem hiding this comment.
can include a link to the code with the example
| 4. the operation is then communicated to all other clients | ||
|
|
||
| ### The document server wants to propagate operations | ||
| If you remember previously how it was mentioned that the document server "propagates" operations to the clients? It does that by sending these operations down a channel maintained by each client |
There was a problem hiding this comment.
rewrite sentence, doesn't make sense
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds documentation for the OT editor 😎