Merge main into feature/update-docs-for-jayvee-updating

.eslintrc.json

@@ -43,10 +43,15 @@
     },
     {
       "files": ["*.spec.ts", "*.spec.tsx", "*.spec.js", "*.spec.jsx"],
+      "plugins": ["jest"],
       "env": {
         "jest": true
       },
-      "rules": {}
+      "rules": {
+        // you should turn the original rule off *only* for test files
+        "@typescript-eslint/unbound-method": "off",
+        "jest/unbound-method": "error"
+      }
     }
   ]
 }

.github/workflows/check-paths-for-windows.yaml

@@ -0,0 +1,29 @@
+# SPDX-FileCopyrightText: 2023 Friedrich-Alexander-Universitat Erlangen-Nurnberg
+#
+# SPDX-License-Identifier: AGPL-3.0-only
+
+name: CI
+
+on:
+  push:
+    branches:
+      - main
+  pull_request:
+  workflow_dispatch:
+
+jobs:
+  check-paths-for-windows:
+    name: Check paths for Windows
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          fetch-depth: 0
+
+      - uses: actions/setup-node@v3
+        with:
+          node-version: 'lts/*'
+
+      - name: Check paths for Windows
+        run: node ./tools/scripts/check-for-invalid-windows-paths.mjs

README.md

@@ -6,7 +6,9 @@ SPDX-License-Identifier: AGPL-3.0-only
 
 # Jayvee
 
-Jayvee is a DSL to model and execute automated data pipelines, e.g., for data engineering. 
+Jayvee is a domain-specific language (DSL) for automated processing of data pipelines.
+The Jayvee interpreter allows executing such data pipelines on local machines.
+Data engineers can use Jayvee and its interpreter to clean and preprocess data for later activities like data science or machine learning.
 
 [![Official Docs](assets/docs-banner.png)](https://jvalue.github.io/jayvee)
 
@@ -26,23 +28,20 @@ In case you would like to contribute to Jayvee, please have a look at our [contr
 
 In case you run into problems, make sure to use the current LTS version of Node.js and npm.
 
-
 ## Projects overview
 
-| Name                                                              | Description                                                                                                                      | NPM package                                |
-|-------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------| ------------------------------------------ |
-| [`language-server`](./libs/language-server)                       | Jayvee language definition and language server implementation                                                                    | [@jvalue/jayvee-language-server](https://www.npmjs.com/package/@jvalue/jayvee-language-server) |
-| [`interpreter`](./apps/interpreter)                               | Command line tool for interpreting Jayvee files                                                                                  | [@jvalue/jayvee-interpreter](https://www.npmjs.com/package/@jvalue/jayvee-interpreter) |
+| Name                                                              | Description                                                                                                                      | NPM package                                                                                                          |
+| ----------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- |
+| [`language-server`](./libs/language-server)                       | Jayvee language definition and language server implementation                                                                    | [@jvalue/jayvee-language-server](https://www.npmjs.com/package/@jvalue/jayvee-language-server)                       |
+| [`interpreter`](./apps/interpreter)                               | Command line tool for interpreting Jayvee files                                                                                  | [@jvalue/jayvee-interpreter](https://www.npmjs.com/package/@jvalue/jayvee-interpreter)                               |
 | [`language-server-web-worker`](./apps/language-server-web-worker) | Ready-to-use Jayvee language server, bundled as a [Web Worker](https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API) | [@jvalue/jayvee-language-server-web-worker](https://www.npmjs.com/package/@jvalue/jayvee-language-server-web-worker) |
-| [`vs-code-extension`](./apps/vs-code-extension)                   | Visual Studio Code extension for editing Jayvee files                                                                            | - |
-| [`docs`](./apps/docs)                                             | Website for Jayvee user documentation                                                                                            | - |
-| [`monaco-editor`](./libs/monaco-editor)                           | React component for editing Jayvee files                                                                                         | [@jvalue/jayvee-monaco](https://www.npmjs.com/package/@jvalue/jayvee-monaco) |
-| [`execution`](./libs/execution)                                   | Shared code for Jayvee extensions and the interpreter                                                                            | - |
-| [`extensions/std`](./libs/extensions/std)                         | Standard Jayvee extension consisting of the extensions below                                                                     | - |
-| [`extensions/rdbms`](./libs/extensions/rdbms)                     | Jayvee extension for relational databases                                                                                        | - |
-| [`extensions/tabular`](./libs/extensions/tabular)                 | Jayvee extension for tabular data                                                                                                | - |
-
-
+| [`vs-code-extension`](./apps/vs-code-extension)                   | Visual Studio Code extension for editing Jayvee files                                                                            | -                                                                                                                    |
+| [`docs`](./apps/docs)                                             | Website for Jayvee user documentation                                                                                            | -                                                                                                                    |
+| [`monaco-editor`](./libs/monaco-editor)                           | React component for editing Jayvee files                                                                                         | [@jvalue/jayvee-monaco](https://www.npmjs.com/package/@jvalue/jayvee-monaco)                                         |
+| [`execution`](./libs/execution)                                   | Shared code for Jayvee extensions and the interpreter                                                                            | -                                                                                                                    |
+| [`extensions/std`](./libs/extensions/std)                         | Standard Jayvee extension consisting of the extensions below                                                                     | -                                                                                                                    |
+| [`extensions/rdbms`](./libs/extensions/rdbms)                     | Jayvee extension for relational databases                                                                                        | -                                                                                                                    |
+| [`extensions/tabular`](./libs/extensions/tabular)                 | Jayvee extension for tabular data                                                                                                | -                                                                                                                    |
 
 ## Scripts
 

apps/docs/docs/dev/03-architecture-overview.md

@@ -12,6 +12,14 @@ On the pure language side, the central project is the [language server](https://
 It contains the syntax definition (i.e. the grammar) and is capable of performing static semantic analysis on models, so invalid models can be rejected and errors are reported to the user.
 It uses the [Language Server Protocol](https://microsoft.github.io/language-server-protocol/) (LSP) for communicating with IDEs in order to provide common features such as diagnostics, auto completion and much more.
 
+**Note:** The [Langium framework](https://langium.org/) generate TypeScript files for the abstract syntax tree (AST), based on the grammar specification.
+The following locations might be especially helpful to understand the grammar and its AST:
+
+- The Langium grammar files (see [here](https://github.com/jvalue/jayvee/tree/main/libs/language-server/src/grammar) or locally at `libs/language-server/src/grammar`; with `.langium` file ending). These files define the **syntax of the language**.
+- The generated TypeScript AST files (execute `npm run generate` to generate them at `libs/language-server/src/lib/ast/generated` in your local repository). These files include **TypeScript interfaces for AST nodes** (e.g., `BlockDefinition`) and **guard methods** (e.g., `isBlockDefinition`).
+  They reflect the input of the grammar files regarding naming.
+- The remaining source files of the language server implement the language server protocol (LSP) and the additional validations beyond the syntax of Jayvee.
+
 ## Interpreter
 
 The Jayvee [interpreter](https://github.com/jvalue/jayvee/tree/main/apps/interpreter) on the other hand is capable of running Jayvee models.

apps/docs/docs/dev/04-guides/02-working-with-the-ast.md

@@ -141,7 +141,7 @@ assert(referenced !== undefined);
 ## AST wrapper classes
 
 The generated interfaces for AST nodes in `ast.ts` are only meant to represent the AST structurally, they don't define any behavior.
-Also, in case of syntactic sugar, there may be different kinds of AST nodes representing the same semantic language concept (e.g. single pipes with a verbose syntax or chained pipes). 
+Also, in case of syntactic sugar, there may be different kinds of AST nodes representing the same semantic language concept. 
 
 To cope with this problem, there is the concept of an `AstNodeWrapper`.
 An AST node wrapper is capable of wrapping AST nodes that represent the same semantic language concept and adding behavior to them via custom methods.

apps/docs/docs/user/composite-blocks.md

@@ -102,7 +102,7 @@ If the CSVExtractor is available in the scope of the `CarsPipeline` from before
 ```jayvee
 pipeline CarsPipeline {
     // HttpExtractor, TextFileInterpreter and CSVInterpreter have been replaced by CSVExtractor
-    CSVExtractor
+    CarsExtractor
         -> CarsTableInterpreter
         -> CarsLoader;
 

apps/docs/docs/user/core-concepts.md

@@ -24,28 +24,6 @@ pipeline CarsPipeline {
 }
 ```
 
-Alternatively, you can use a slightly longer syntax for pipes:
-
-```jayvee
-pipeline CarsPipeline {
-    // Assumption: blocks "GasReserveHttpExtractor", "GasReserveCSVInterpreter", "GasReserveTableInterpreter", and "GasReserveLoader" are defined
-
-    pipe {
-        from: GasReserveHttpExtractor;
-        to: GasReserveTextFileInterpreter;
-
-    }
-
-    pipe {
-        from: GasReserveTextFileInterpreter;
-        to: GasReserveCSVInterpreter;
-
-    }
-
-    // etc.
-}
-```
-
 ## Blocks
 
 A `Block` is a processing step within a `Pipeline`.

apps/docs/docs/user/examples/.gitignore

@@ -2,4 +2,5 @@
 #
 # SPDX-License-Identifier: AGPL-3.0-only
 
+!README.mdx
 *.md

apps/docs/docs/user/examples/README.mdx

@@ -0,0 +1,15 @@
+---
+sidebar_position: 10
+---
+
+# Jayvee Examples
+
+Examples of Jayvee models.
+Copy them to your local file system and execute them with the `jv` command on your command line (see [usage](/docs/user/intro#usage)).
+You can [find all examples on Github](https://github.com/jvalue/jayvee/tree/main/example).
+
+```mdx-code-block
+import DocCardList from '@theme/DocCardList';
+
+<DocCardList />
+```

apps/docs/versioned_docs/version-0.3.0/dev/01-intro.md

@@ -0,0 +1,52 @@
+---
+sidebar_position: 1
+---
+
+# Introduction for Jayvee Developers
+
+## How to contribute
+
+In order to contribute to the Jayvee project, please have a look at our [contribution guide](https://github.com/jvalue/jayvee/blob/main/CONTRIBUTING.md). Before planning a contribution, please read the [design principles](./05-design-principles.md) and consider if your changes fit the vision expressed there.
+
+The overall project is licensed under the `AGPL-3.0-only` license and is compliant to the [REUSE Specification](https://reuse.software/spec/) by the [Free Software Foundation Europe](https://fsfe.org/).
+Because of this, contributions are required to adhere to the license and follow that specification.
+More details on this topic can be found [here](./02-dev-processes/03-licensing-and-copyright.md).
+
+And last but not least, please read and follow our [code of conduct](https://github.com/jvalue/jayvee/blob/main/CODE_OF_CONDUCT.md).
+
+## Project overview
+
+The Jayvee repository is located [here](https://github.com/jvalue/jayvee) on GitHub.
+It uses an [Nx mono-repository](https://nx.dev/) setup and contains all related projects, most notably the Jayvee language server and interpreter.
+Please have a look at the [architecture overview](./03-architecture-overview.md) for more details.
+
+## Prerequisites
+
+Node.js and npm are required for development.
+It is recommended to use their LTS version to avoid any potential compatibility issues.
+Also, refer to this [quick start guide](https://github.com/jvalue/jayvee#development-quickstart) for developers.
+
+In order to run the Jayvee VS Code extension during development, VS Code is required as IDE.
+Using VS Code also allows installing the [Langium VS Code extension](https://marketplace.visualstudio.com/items?itemName=langium.langium-vscode) for better IDE support when working with Langium grammar files.
+
+## Resources for getting started
+
+### Langium
+
+- [**Langium documentation**](https://langium.org/docs/)
+- Practical examples using Langium:
+  - [Langium examples](https://github.com/langium/langium/tree/main/examples): Official example languages by Langium
+  - [Langium's grammar language](https://github.com/langium/langium/tree/main/packages/langium): The implementation of Langium's own grammar language
+  - [Language server for SQL](https://github.com/langium/langium-sql): An implementation of a language server for SQL
+  - [MiniLogo DSL](https://github.com/langium/langium-minilogo): A domain-specific language for drawing pictures
+  - [Lox language](https://github.com/langium/langium-lox): An implementation of the Lox scripting language (also see the "Crafting Interpreters" book below)
+  - [SimpleUI DSL](https://github.com/TypeFox/langium-ui-framework): A domain-specific language for generating user interfaces
+  - [STPA-DSL](https://github.com/kieler/stpa): A domain-specific language for System-Theoretic Process Analysis
+- [Langium playground](https://langium.org/playground/): A tool for testing Langium grammars and visualizing resulting ASTs
+- [Typefox blog](https://www.typefox.io/blog/): A blog by the company behind Langium, mostly posting Langium-related content.
+
+### Building compilers / interpreters / DSLs
+
+- [Crafting Interpreters](https://craftinginterpreters.com/contents.html): A book by Robert Nystrom on implementing an interpreter for the Lox scripting language
+- [DSL Engineering](https://voelter.de/dslbook/markusvoelter-dslengineering-1.0.pdf): A book by Markus Voelter on designing, implementing and using domain-specific languages
+- [Awesome Compilers](https://github.com/aalhour/awesome-compilers#readme): A list featuring many resources on compilers and interpreters

apps/docs/versioned_docs/version-0.3.0/dev/01-intro.md.license

@@ -0,0 +1,3 @@
+SPDX-FileCopyrightText: 2023 Friedrich-Alexander-Universitat Erlangen-Nurnberg
+
+SPDX-License-Identifier: AGPL-3.0-only

apps/docs/versioned_docs/version-0.3.0/dev/02-dev-processes/01-rfc-process.md

@@ -0,0 +1,13 @@
+---
+title: Language Design Process (RFCs)
+sidebar_position: 1
+---
+
+We use RFCs to discuss changes to the language before implementing them. You can have a look at all closed (accepted / rejected) RFCs [here](https://github.com/jvalue/jayvee/pulls?q=is%3Apr+is%3Aclosed+RFC+), and all RFCs under discussion [here](https://github.com/jvalue/jayvee/pulls?q=is%3Apr+is%3Aopen+RFC).
+
+If you want to contribute an RFC please follow these steps:
+1. Make a copy of the **template** at `rfc/0000-rfc-template.md` and place it into the `rfc` folder.
+2. Create a draft for the RFC on a new branch. Follow the `TODOs` in template to do so.
+3. Open a pull request with prefix `RFC <number>` in the title.
+4. Address the reviews. Consider opening a new PR if major things need to be addressed and the discussion log becomes too confusing.
+5. Once accepted, create an issue with the necessary steps to implement the RFC.

apps/docs/versioned_docs/version-0.3.0/dev/02-dev-processes/01-rfc-process.md.license

@@ -0,0 +1,3 @@
+SPDX-FileCopyrightText: 2023 Friedrich-Alexander-Universitat Erlangen-Nurnberg
+
+SPDX-License-Identifier: AGPL-3.0-only

apps/docs/versioned_docs/version-0.3.0/dev/02-dev-processes/02-debug-vs-code-extension.md

@@ -0,0 +1,25 @@
+---
+title: Debugging via the VS Code extension
+sidebar_position: 2
+---
+
+The VS Code extension of Jayvee can be used to interactively debug the language server.
+During development, when using VS Code as IDE, another instance of VS Code can be opened which has the most recent, locally built VS Code extension loaded.
+This can be achieved using the launch configuration `Run extension` the `Run and Debug` side-menu of VS Code or by pressing the `F5` key if that launch configuration is already selected there.
+
+Note that there is no file watching mechanism involved.
+So in order to reflect changes to the source code, the additional VS Code instance has to be **closed and reopened** as described above.
+
+## How to attach a debugger
+
+1. Use the launch configuration `Run extension` to open the additional VS Code instance with the extension loaded
+2. Use the launch configuration `Attach to Language Server` to attach the debugger
+
+Any set breakpoints should now be marked as active and pause the execution once they are reached.
+
+## How to view logs of the language server
+
+In the additional VS Code instance, it is possible to view the logs of the language server.
+They might also be helpful for debugging since any uncaught errors are logged with their stack trace by the Langium framework.
+
+To view the logs, open the bottom panel called `Output` and select `Jayvee` in the dropdown menu.

apps/docs/versioned_docs/version-0.3.0/dev/02-dev-processes/02-debug-vs-code-extension.md.license

@@ -0,0 +1,3 @@
+SPDX-FileCopyrightText: 2023 Friedrich-Alexander-Universitat Erlangen-Nurnberg
+
+SPDX-License-Identifier: AGPL-3.0-only