From 877a780732850d0eab8bba355bd8895d9ed09afc Mon Sep 17 00:00:00 2001 From: Haile Lagi <52631736+hailelagi@users.noreply.github.com> Date: Sat, 7 Dec 2024 22:02:41 +0100 Subject: [PATCH] fs simple layout Signed-off-by: Haile Lagi <52631736+hailelagi@users.noreply.github.com> --- content/notes/block-layers.md | 64 +++++++++++++++++++++++++++++++++++ content/notes/filesystems.md | 14 -------- 2 files changed, 64 insertions(+), 14 deletions(-) create mode 100644 content/notes/block-layers.md delete mode 100644 content/notes/filesystems.md diff --git a/content/notes/block-layers.md b/content/notes/block-layers.md new file mode 100644 index 0000000..e3d0386 --- /dev/null +++ b/content/notes/block-layers.md @@ -0,0 +1,64 @@ +--- +title: "through the looking glass of block layers" +date: 2024-12-06T17:38:16+01:00 +draft: true +--- + +The modern computing/data infrastructure is [vast and interesting](https://landscape.cncf.io/). +Let's explore a tiny slice of it, what happens when you read or write some data **persistently** on a modern cloud provider? +Let's conceptually trace our way down the block layers and see where data goes by writing a filesystem ??? + +{{% callout %}} +All problems in comp sci. can be solved by another level of indirection. +{{% /callout %}} + +Why a filesystem? It's **a key abstraction** we'll use to go spelunking and of course we'll spin up an ec2 instance + and mount a filesystem for it with s3! + + +## Hardware +### Hard disk drive +Exposes a "flat" address space to read or write. Smallest atomic unit is a sector (512-byte block). + +### SSD +flash/cells. + +### Problems +- Latent sector errors +- Misdirected IO +- Disk corruption (physical media - heat etc) + + +### Disk IO scheduling/schedulers +- SSTF +- NBF +- SCAN vs C-SCAN (elevator algorithm) +- SPTF + +linux: https://wiki.ubuntu.com/Kernel/Reference/IOSchedulers + +### RAID +transparently map logical IO to physical IO for fault-tolerance(fail-stop model) and performance. +- stripping +- mirroring +- parity + +### File system +An OS interface/sub-system that allows the management of blocks + block devices on disk via abstractons: + +``` +++++++++++++++++++++++++++++++++++++++++++++ ++ superblock + inode-table + user data! +++ +++++++++++++++++++++++++++++++++++++++++++++ +``` + +## Data structures: +1. the file (INode + ) +2. The directory (self `.`, parent `..`, etc) +3. access methods: open(), read(), write() +4. super block - metadata about other metadata (inode count, fs version, etc) + +## Design choices/tradeoffs +- Tree vs Array +- Bitmap index vs free list vs Btree +- Indexing non-contiguous layout (pointers vs extents) +- static vs dynamic partitioning diff --git a/content/notes/filesystems.md b/content/notes/filesystems.md deleted file mode 100644 index b729496..0000000 --- a/content/notes/filesystems.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -title: "Filesystems" -date: 2024-12-06T17:38:16+01:00 -draft: true ---- - -A simple filesystem, key ideas, tradeoffs. -Flubber, a fuse. -Production filesystems. - -## Data structures: -- INode -- array of blocks vs trees -- access methods: open(), read(), write()