Reorganize the repository layout

.github/workflows/build.yml

@@ -12,8 +12,8 @@ jobs:
       - uses: actions/checkout@v2
         with:
           fetch-depth: 0
-      - run: sudo apt install tup yasm
-      - run: tup
+      - run: sudo apt install yasm
+      - run: ./build.sh
       - run: python3 .github/workflows/name_release.py
         id: name_release
       - uses: softprops/action-gh-release@v1

.gitignore

@@ -0,0 +1,4 @@
+*.bin
+*.lst
+*.img
+__pycache__/

README.md

@@ -16,33 +16,10 @@ The dictionary is case-sensitive. If a word is not found, it is converted into a
 with no error checking. For example, `g` results in the decimal 16, extending
 the `0123456789abcdef` of hexadecimal. On boot, the number base is set to hexadecimal.
 
-Backspace works, but doesn't erase the input with spaces, so until you write something else,
-the screen will look a bit weird.
+Backspace works, but not how you're used to — the erased input will be still visible on
+screen until you write something else.
 
-The main goal of the project is bootstrapping a full system on top of Miniforth
-as a seed. I describe this in more details [on my blog][blog].
-
-## Blocks
-
-`load ( blk -- )` loads a 1K block of FORTH source code from disk and executes it.
-All other block operations are deferred to user code. Thus, after appropriate setup,
-one can get an arbitrarily feature-rich system by simply typing `1 load`.
-
-Each pair of sectors on disk forms a single block. Block number 0 is partially used
-by the MBR, and is thus reserved.
-
-## System variables
-
-Due to space constraints, variables such as `STATE` or `BASE` couldn't be exposed by creating
-separate words. Depending on the variable, the address is either hardcoded or pushed onto
-the stack on boot:
-
- - `>IN` is a word at `0xa02`. It stores the pointer to the first unparsed character
-   of the null-terminated input buffer.
- - The stack on boot is `LATEST STATE BASE HERE #DISK` (with `#DISK` on top).
- - `STATE` has a non-standard format - it is a byte, where `0` means compiling,
-   and `1` means interpreting.
- - `#DISK` is not a variable, but the saved disk number of the boot media
+*Various aspects of this project's internals are described in detail [on my blog][blog].*
 
 ## Trying it out
 
@@ -53,46 +30,29 @@ When Miniforth boots, no prompt will be shown on the screen. However, if what
 you're typing is being shown on the screen, it is working. You can:
 
  - do some arithmetic: `1 2 + u.`
- - load the code I've developed on top of Miniforth: `1 load`
-   - browse that code: `7 21 index`, `7 1000 read-block`, `1000 list`
+ - load additional functionality from disk: `1 load`
+   (see [*Onwards from miniforth*](#onwards-from-miniforth) below).
 
 ## Building a disk image
 
-If you have Nix installed, get the build dependencies with `nix-shell`.
-Otherwise, you will need `yasm`, `python3`, and optionally, `tup`
-(Tup is used instead of Make for Tup's automatic dependency management,
-which would be a pain for `mkdisk.py`).
-
-Run
-
-```
-tup
-```
-
-A non-incremental build can be done without `tup` with the following commands:
-
-```
-yasm -f bin boot.s -o raw.bin -l boot.lst
-python3 compress.py
-python3 mkdisk.py
-```
+You will need `yasm` and `python3`, which you can obtain with `nix-shell` or
+your package manager of choice. Then run `./build.sh`.
 
 This will create the following artifacts:
 
 - `boot.bin` - the built bootsector.
+- `uefix.bin` - the chainloader (see below).
 - `disk.img` - a disk image with the contents of `block*.fth` installed into
   the blocks.
 - `boot.lst` - a listing with the raw bytes of each instruction.
    Note that the `dd 0xdeadbeef` are removed by `compress.py`.
 
 The build will print the number of used bytes, as well as the number of block files found.
 You can run the resulting disk image in QEMU with `./run.sh`, or pass `./run.sh boot.bin`
-if you do not want to include the blocks in your disk. QEMU will run in curses mode, exit
-with <kbd>Alt</kbd> + <kbd>2</kbd>, <kbd>q</kbd>, <kbd>Enter</kbd>.
+if you do not want to include the code from `*.fth` in your disk. QEMU will run in curses
+mode, exit with <kbd>Alt</kbd> + <kbd>2</kbd>, <kbd>q</kbd>, <kbd>Enter</kbd>.
 
-## Getting started
-
-## Hardware considerations
+## UEFI sucks, or `uefix.bin`
 
 Running Miniforth on real hardware is certainly possible. In fact, I hardly use
 emulation for it these days. Anything not unreasonably old should work. However,
@@ -101,14 +61,95 @@ partition table of an MBR before booting from it. As Miniforth does not include
 a partition table in the bootsector (as there is simply no space for it), this
 prevents booting on most UEFI computers.
 
-To remedy this, this repository includes a small chainloader to fix it —
-`uefix.s`. While running `tup` compiles it, you can also do that manually with
+To remedy this, this repository includes a small chainloader as a workaround.
+So, instead of this disk layout, which works on computers old enough:
+
+```
+LBA 0   - boot.bin
+LBA 1   - unused
+LBA 2-3 - Forth block 1
+...       ...
+```
+
+...the disk image generated by `build.sh` looks as follows:
 
-```bash
-yasm -f bin uefix.s -o uefix.bin
 ```
+LBA 0   - uefix.bin
+LBA 1   - boot.bin
+LBA 2-3 - Forth block 1
+...       ...
+```
+
+## Blocks
 
-Then, install `uefix.bin` in sector 0, and `boot.bin` in sector 1.
+`load ( blk -- )` loads a 1K block of FORTH source code from disk and executes it.
+All other block operations are deferred to user code. Thus, after appropriate setup,
+one can get an arbitrarily feature-rich system by simply typing `1 load` —
+see [*Onwards from miniforth*](#onwards-from-miniforth) below.
+
+Each pair of sectors on disk forms a single block. Block number 0 is partially used
+by the MBR, and is thus reserved.
+
+## System variables
+
+Due to space constraints, variables such as `STATE` or `BASE` couldn't be exposed by creating
+separate words. Depending on the variable, the address is either hardcoded or pushed onto
+the stack on boot:
+
+ - `>IN` is a word at `0xa02`. It stores the pointer to the first unparsed character
+   of the null-terminated input buffer.
+ - The stack on boot is `LATEST STATE BASE HERE #DISK` (with `#DISK` on top).
+ - `STATE` has a non-standard format - it is a byte, where `0` means compiling,
+   and `1` means interpreting.
+ - `#DISK` is not a variable, but the saved disk number of the boot media
+
+## Onwards from miniforth
+
+The main goal of the project is bootstrapping a full system on top of Miniforth
+as a seed. Thus the repository also contains various Forth code that may run on
+top of Miniforth and extend its capabilities.
+
+ - In `bootstrap.fth` (`1 load`):
+   - A simple assembler is implemented, and then used to implement additional
+     primitives, which wouldn't fit in Miniforth itself. This includes control
+     flow words like `IF`/`THEN` and `BEGIN`/`UNTIL`, as well as calls to the BIOS
+     disk interrupt to allow manipulating the code on disk.
+
+     For the syntax of the assembler, see [*No branches? No problem — a Forth
+     assembler*][branch-blog].
+
+   - Exception handling is implemented, with semantics a little different from
+     standard Forth. See [*Contextful exceptions with Forth
+     metaprogramming*][exception-blog].
+   - A separate, more featureful outer interpreter overrides the one built into
+     Miniforth, to correct the ugly backspace behavior and handle things
+     such as uncaught exceptions and vocabularies.
+   - A way of searching for occurences of a particular string in the code stored
+     in the blocks is provided:
+     - `10 20 grep create` searches blocks `$10` through `$20` inclusive for
+       occurences of `create`
+     - If your search term includes spaces, use `grep"` — the syntax is similar
+       to `s"` string literals: `10 20 grep" : >number"`
+ - In `editor.fth` (`30 load`), a vi-like block editor is implemented. It can be started
+   with e.g. `10 edit` to edit block 10.
+   - Non-standard keybindings:
+     - <kbd>Q</kbd> to quit back to the Forth REPL.
+     - <kbd>[</kbd> to look at the previous block.
+     - <kbd>]</kbd> to look at the next block.
+   - After first use, you can use the shorthand `ed` to reopen the last-edited block.
+   - Use `run` to execute the last-edited block. This sets a flag to prevent
+     a chain of `-->` from loading all the subsequent blocks.
+   - Changes are saved to disk whenever you use `run` or open a different block with `edit`
+     or the <kbd>[</kbd>/<kbd>]</kbd> keybinds. You can also trigger this
+     manually with `save`.
+
+All this code was originally developed within Miniforth itself, which meant it was
+stored within a disk image — a format that's not very friendly to tooling like
+Git or GitHub's web interface. This disparity is handled by two Python scripts:
+
+ - `mkdisk.py` takes the files and merges them into a bootable disk image;
+ - `splitdisk.py` extracts the code from a disk image's blocks and splits
+   it into files.
 
 ## Free bytes
 
@@ -142,4 +183,6 @@ If a feature is strongly desirable, potential tradeoffs include:
 
 [FORTH]: https://en.wikipedia.org/wiki/Forth_(programming_language)
 [blog]: https://compilercrim.es/bootstrap/
+[branch-blog]: https://compilercrim.es/bootstrap/branches/
+[exception-blog]: https://compilercrim.es/bootstrap/exception-context/
 [the releases page]: https://github.com/meithecatte/miniforth/releases/