feat: basic leanpub generation (#317)

* feat: basic leanpub generation

* fix: correct links

* build: deploy preview on PR

* fix: correct URLs

* build: update checkout actions

* build: lightweight build for preview

* build: fix build

* build: fix build

* build: fix build

* build: fix build

.github/workflows/pull-request.yaml

@@ -8,7 +8,7 @@ jobs:
     runs-on: ubuntu-20.04
     steps:
     - name: Checkout
-      uses: actions/checkout@v3
+      uses: actions/checkout@v4
 
     # Fixup Git URLs, see:
     # https://stackoverflow.com/questions/70663523/the-unauthenticated-git-protocol-on-port-9418-is-no-longer-supported
@@ -22,3 +22,20 @@ jobs:
     # Run the build - this will fail if there are broken links etc.
     - name: Build
       run: make build
+
+  # See:
+  # https://github.com/marketplace/actions/deploy-pr-preview
+  deploy-preview:
+    runs-on: ubuntu-20.04
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Install and Build
+        if: github.event.action != 'closed' # You might want to skip the build if the PR has been closed
+        run: npm install --legacy-peer-deps && npm run build
+
+      - name: Deploy preview
+        uses: rossjrw/pr-preview-action@v1
+        with:
+          source-dir: ./build/

.github/workflows/release-please.yaml

@@ -27,7 +27,7 @@ jobs:
       # deploy the website.
       # Note: *every* step here needs to check the 'release_created' flag.
       - name: Checkout
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
         if: ${{ steps.release.outputs.release_created }}
 
       # Fixup Git URLs, see:

TODO-donate.md

@@ -0,0 +1,52 @@
+## TODO: Structure for Jill and Bill
+
+three bullet summary - really emphasise the practical chapters
+1. by definition starts off slow
+2. gets really practical on text manip
+3. slows a bit for scripts
+4. then really digs into making the environment your ownhttps://effective-shell.com/
+
+- [ ] Chapter by chapter go through the outline; pass that to Bill + Jill
+- [ ] each chapter should be useful hands on experiments
+- [ ] each chapter what is the differentiator and fresh skill you will learn
+- [ ] clear summary for Bill and Jill - who the book is for, the intro
+
+## Lean=pub
+
+
+{sample: true, community: true}
+
+## TODO monday's blast out
+
+ycombinator
+r/programming
+linkedin
+mckinsey slacks
+
+## TODO: Donate
+
+- [ ] Create donate button
+- [ ] Create donate page
+- [ ] Create cancel page
+- [ ] Create thanks page
+- [ ] Create share page
+- [ ] Test
+- [ ] Add GA funnel and goals
+- [ ] Setup Task for integrated button (on page experience?)
+
+No need for leanpub - it generates a manu and I have that
+However, markdown to manu could easily generate text for leanpub, most of leanpub is just markdown, except for 
+
+{aside category='warning'}
+whatever
+{/aside}
+
+However, why bother with this, if it's online reading people will want, just create a donate page and offer a PDF download for minimum donations.
+
+- Donation could be a sliding bar, showing how much the author gets
+- Create a donate page, suggested minimum and maximum
+- track this as a goal in GA
+- share the book more widely
+- integrate 'donate' links or a 'donate' popup some time
+- collect feedback when donating
+- [ ] leanpub style slider: https://codesandbox.io/s/leanpub-pricing-slider-clone-8mr9692nl?file=/src/Slider/index.js

docs/index.mdx → docs/00-index.mdx

@@ -1,19 +1,23 @@
 ---
 title: Effective Shell
+leanpub_title: Introduction
+leanpub_header: |-
+  {sample: true}
+  {class: part}
 sidebar_position: 1
 sidebar_label: Introduction
 slug: '/'
 ---
 
-import { EmailSignupForm } from '@site/src/components/EmailSignupForm/EmailSignupForm';
-
 Investing just a few hours in your ability to use text based interfaces for your computer can have an enormous impact on your productivity. It can also make your work more fun, allowing you to maintain that creative 'flow' state that can make technology so exciting.
 
 I've been lucky enough to spend many years working as a software engineer, but also with scientists, data engineers, site reliability engineers and technologists of all sorts. One thing that has stood out about great technologists has been their ability to make their tools work for them, stitching them together in creative ways that suits their style.
 
 This is not a book on shell scripting or Linux administration! Each chapter in this book can be read as a standalone set of techniques to help you be more efficient, understand your system in more depth, and craft your environment to suit your working style. This book does not advocate that you totally change the way you work or drop your current tooling; it just brings together a set of skills that you can add to your toolkit and incorporate in your own way.
 
+<!-- manuscript:ignore:start -->
 If you find this book useful, please consider [donating](/donate) to support my open source work.
+<!-- manuscript:ignore:end -->
 
 ## Who Should Read This Book
 
@@ -27,12 +31,12 @@ Hobbyists, polymaths and explorers should also read on - as well as going into s
 
 Each chapter of this book aims to work as a stand-alone description of a set of techniques that you should be able to apply immediately. I have focused on keeping the information to the essentials that allow you to use the skill, rather than create an exhaustive description of every possible feature. This should allow you to pick up the book and read a chapter over a coffee and try out the skills straight away.
 
- - [_Part 1 - Transitioning to the Shell_](./01-transitioning-to-the-shell/index.md) is aimed at people who are new to the shell. You'll learn how to set up your environment, navigate your system, manage files, move between a desktop environment and the shell and how to get help. More advanced users may want to skip some of this content.
- - [_Part 2 - Core Skills_](./02-core-skills/index.md) introduces the essential techniques that you should be able to apply immediately to improve your productivity. You'll learn how to use pipelines, rapidly move around the command-line and commands quickly, manage multiple tasks in parallel with jobs, the different types of commands that are available and how to search for files.
- - [_Part 3 - Manipulating Text and Streams_](./03-manipulating-text/index.md) demonstrates many techniques to work with text -whether it's code, data or configuration. You'll learn how to use regular expressions, how to search through text, how to slice and dice text, how to manipulate code and data, and then how to apply these techniques to your shell commands themselves.
- - [_Part 4 - Shell Scripting_](./04-shell-scripting/index.md) is a crash course in shell scripting. You'll learn how to write and run scripts, read and process input, perform logical operations, iterate over files and folders, build functions, handle errors and a set of re-usable patterns you can apply to your own scripts.
- - [_Part 5 - Building Your Toolkit_](./05-building-your-toolkit/index.md) goes into the techniques you can use to create and customize your own environment. You'll learn how to configure the shell, customize your command prompt, manage your configuration files and share and manage your configuration with the Git version control system.
- - [_Part 6 - Advanced Techniques_](./06-advanced-techniques/index.md) introduces even more powerful tools and techniques, showing the fundamentals of each skill and how you might incorporate it into your work. You'll learn how shell expansion works, which is key to understanding many of the nuances of complex commands, when you should move away from shell scripts and into a full-blown programming language, how to build programs that integrate well into other tools in the shell, how to connect to other machines with the secure shell, how to use the popular terminal editor Vim and how to use terminal multiplexers. 
+ - [_Part 1 - Transitioning to the Shell_](./01-transitioning-to-the-shell/00-index.md) is aimed at people who are new to the shell. You'll learn how to set up your environment, navigate your system, manage files, move between a desktop environment and the shell and how to get help. More advanced users may want to skip some of this content.
+ - [_Part 2 - Core Skills_](./02-core-skills/00-index.md) introduces the essential techniques that you should be able to apply immediately to improve your productivity. You'll learn how to use pipelines, rapidly move around the command-line and commands quickly, manage multiple tasks in parallel with jobs, the different types of commands that are available and how to search for files.
+ - [_Part 3 - Manipulating Text and Streams_](./03-manipulating-text/00-index.md) demonstrates many techniques to work with text -whether it's code, data or configuration. You'll learn how to use regular expressions, how to search through text, how to slice and dice text, how to manipulate code and data, and then how to apply these techniques to your shell commands themselves.
+ - [_Part 4 - Shell Scripting_](./04-shell-scripting/00-index.md) is a crash course in shell scripting. You'll learn how to write and run scripts, read and process input, perform logical operations, iterate over files and folders, build functions, handle errors and a set of re-usable patterns you can apply to your own scripts.
+ - [_Part 5 - Building Your Toolkit_](./05-building-your-toolkit/00-index.md) goes into the techniques you can use to create and customize your own environment. You'll learn how to configure the shell, customize your command prompt, manage your configuration files and share and manage your configuration with the Git version control system.
+ - [_Part 6 - Advanced Techniques_](./06-advanced-techniques/00-index.md) introduces even more powerful tools and techniques, showing the fundamentals of each skill and how you might incorporate it into your work. You'll learn how shell expansion works, which is key to understanding many of the nuances of complex commands, when you should move away from shell scripts and into a full-blown programming language, how to build programs that integrate well into other tools in the shell, how to connect to other machines with the secure shell, how to use the popular terminal editor Vim and how to use terminal multiplexers. 
 
 For the newcomer, you'll learn what a shell is, how to use it on your system, and then how to become more effective everyday by integrating the shell into your work. For the experienced professional, there is a wealth of detailed tips and tricks in each chapter that go into advanced topics and techniques to make you even more of a power user.
 
@@ -61,8 +65,12 @@ Z shell specifics are highlighted like this.
 
 :::
 
+<!-- manuscript:ignore:start -->
 ## Email Updates
 
 If you would like to get email updates when new chapters are published, please do provide your email below. I won't be using it for anything beyond updates to the book.
 
+import { EmailSignupForm } from '@site/src/components/EmailSignupForm/EmailSignupForm';
 <EmailSignupForm />
+
+<!-- manuscript:ignore:end -->

docs/01-transitioning-to-the-shell/index.md → docs/01-transitioning-to-the-shell/00-index.md

@@ -1,6 +1,10 @@
 ---
 title: 'Part 1 - Transitioning to the Shell'
 slug: '/part-1-transitioning-to-the-shell/'
+leanpub_header: |-
+  {mainmatter}
+  {sample: true}
+  {class: part}
 ---
 
 These are the key skills which everyone should know. Without them, you might struggle to perform certain tasks at all. Experienced users can probably skip this section, or just review the summary. But if you are new to the shell, this is the best place to start! This section focuses on helping you quickly get up to speed with how to perform the same kind of tasks you might have performed in a GUI (Graphical User Interface) with the shell.

docs/02-core-skills/index.md → docs/02-core-skills/00-index.md

@@ -1,6 +1,9 @@
 ---
 title: 'Part 2 - Core Skills'
 slug: '/part-2-core-skill/'
+leanpub_header: |-
+  {sample: true}
+  {class: part}
 ---
 
 In the first part of this book we look at the shell from the perspective of someone who is familiar with a graphical user interface. We studied how to transition from a GUI to the shell, introducing the 'shell way' of performing tasks which you might have previously performed using a GUI.

docs/04-shell-scripting/19-variables-reading-input-and-mathematics/index.md

@@ -436,7 +436,7 @@ Notice the similarity to the Array operators such as `${#array[@]}` to get the l
 
 There are a number of other operators that exist. They allow you to extract parts of a string, apply regular expressions, manipulate the case and perform a number of complex operations. I would avoid these techniques if possible as they are fairly specific to Bash and likely will be confusing to readers. Some of these substitutions are not available in older versions of Bash.
 
-If you need to manipulate text I would recommend that you use the techniques described in [**Part 3 - Manipulating Text**](../../03-manipulating-text/index.md).
+If you need to manipulate text I would recommend that you use the techniques described in [**Part 3 - Manipulating Text**](../../03-manipulating-text/00-index.md).
 
 It is generally enough to know that if you see special symbols inside a `${variable}` expression then the writer is performing some kind of string manipulation. Hopefully they have included a comment that describes what they are doing to make it easier to follow!
 

docs/xx-appendices/exercises.md

@@ -0,0 +1,6 @@
+# Good Scripts to write as exercises
+
+- `recent` - a better version of `history`, which deduplicates and sorts based on the most commonly used items
+- `quickman` - a quick link to the most common man pages
+- `options` - show and interactively toggle options
+- `phonetic` - take a sequence of letters/numbers and spell it out using NATO phonetic alphabet

docs/xx-appendices/the-future.md

@@ -0,0 +1,7 @@
+The shell, in particular the Bourne-Again Shell has been popular for many years. But what does the future hold? With the advent of the Linux Subsystem for Windows, new shells like Nushell, and the latest version of MacOS switching from Bash to Z-Shell, we finish off by looking at some of the trends which might shape how we use shells in the future.
+
+Scattered notes so far:
+
+- [A Terminal for the Graphical Age](https://medium.com/the-graphical-terminal/a-terminal-for-the-graphical-age-87ab7aaa2749) - Great article on an ASCII terminal with modern features. Very interesting project.
+- Another interesting project is Nu: https://github.com/nushell/nushell
+- There's a great podcast about Nu here: https://changelog.com/podcast/363

docs/zz-developer-guide/markdown-guide.mdx

@@ -0,0 +1,86 @@
+---
+title: Markdown
+---
+
+The contents of this book are primarily written in MDX. Some key concepts to be aware of:
+
+- [MDX](https://github.com/mdx-js/specification) is the language that most files in the `./doc` folder are written in
+- [Commonmark](https://commonmark.org/) is the standard markdown spec that MDX is based on
+- [GitHub Flavored Markdown](https://github.github.com/gfm/) is also based on Commonmark
+- [Markua](https://leanpub.com/markua/read) is used internally for the Leanpub distribution
+
+Essentially this means that for the book to render properly in Docusaurus, preview nicely in GitHub, and be able to be transformed into a format for use with Leanpub, some specific rules should be followed.
+
+## Links
+
+TODO: links should be explicit e.g.  `00-index.md` rather than `index.md`
+
+## Emphasis
+
+TODO: Use *single stars for italics* and **double stars for bold** - because Markua confusingly uses _single underscores for underline_.
+
+## Asides / Admonitions / Blurbs
+
+TODO
+
+## Markua Tags
+
+TODO
+
+`frontmatter` marks content that is considered part of the introduction or preface. It is numbered in the manuscript with numerals such as *i*, *ii*, *xii*.
+
+`mainmatter` indicates content that is part of the main book, it is numbered with numerals.
+
+`backmatter` todo
+
+## Titles
+
+Frontmatter
+
+## Images
+
+TODO image names must be unique
+
+## Images
+
+:::info Not in Use
+
+Ideal Image is not currently enabled as it appears to clash with native Docusaurus lazy-loading.
+
+:::
+
+The [`@docusaurus/plugin-ideal-image`](https://docusaurus.io/docs/api/plugins/@docusaurus/plugin-ideal-image) is used to allow lazy loading and zoom of images.
+
+Use the `Image` tag as shown:
+
+```markdown
+import Image from '@theme/IdealImage';
+<Image img={require('./images/vim-cheatsheet.png')} />
+```
+
+This will render an image as shown below:
+
+::: warn Currently Disabled
+
+```
+<Image img={require('./images/vim-cheatsheet.png')} />
+```
+
+:::
+
+## Diagrams
+
+Render a Draw.io diagram like so:
+
+```
+import Drawio from '@theme/Drawio'
+import asymmetricEncryption from '!!raw-loader!./images/asymmetric-encryption.drawio';
+
+<Drawio content={asymmetricEncryption} />
+```
+
+This would render as below:
+
+import asymmetricEncryption from '!!raw-loader!./images/asymmetric-encryption.drawio';
+
+<Drawio content={asymmetricEncryption} />

scripts/leanpub/.gitignore

@@ -0,0 +1 @@
+*.csv

scripts/leanpub/README.md

@@ -0,0 +1,81 @@
+# leanpub
+
+Limitations:
+
+- Sorting is the natural order of the files, so:
+
+```
+docs/00-index.mdx         # works, comes first
+docs/01-part1/00-index.md # works
+docs/index.mdx            # doesn't work, comes too late
+```
+
+Basic script to test the validity of names of E2E suites and steps, as well as provide some metrics on reports. Can also be used in CI to fail PRs that do not have valid test names.
+
+**This is a script** - it's not production grade code, it's for quick diagnostics and CI processes. It does not have tests, adhere to standard linting patterns and so on. If we want to industrialise this, we should extract it into it's own repo, module, and put a CLI around it.
+
+## Quickstart
+
+Run the script, providing the path to the test files:
+
+From the `Lingo` folder run:
+
+```bash
+npm run e2e-test-report
+```
+
+Or manually run the script, providing the path to the test files:
+
+```bash
+node report.js "../tests/specs"
+# debug with...
+node --inspect --inspect-brk report.js "../tests/specs"
+```
+
+This will:
+
+1. Recursively find all files matching the pattern `*.spec.ts`
+2. Go through each file and validate top-level `describe` calls to ensure that they match the test name requirements
+3. Go through each file and validate the `it` calls to ensure that they match the step name requirements
+4. Write a report to `report.csv` for quick analysis
+5. Write a summary to the console for quick review
+6. Exit wit a non-zero code if any tests or steps do not match the requirements
+
+## Test and Step Requirements
+
+- `\d+_RN\:\d+` - tests have a requirement number and protocol id
+- `^Step\s?[\d\.]+` steps are clearly named and numbered
+
+## CSV Report output
+
+The follow columns are written:
+
+| Column        | Description                                                   |
+| ------------- | ------------------------------------------------------------- |
+| Path          | The path to the test file                                     |
+| ValidProtocol | TRUE if the test name correctly links to a protocol           |
+| ValidSteps    | TRUE if the names of the steps in the test are named properly |
+| NumberOfSteps | The number of steps in the test                               |
+| Name          | The actual name of the test                                   |
+
+## Developer Guide
+
+Format the code with:
+
+```bash
+npx prettier --write
+```
+
+Run tests with:
+
+```bash
+npx jest *.spec.js
+```
+
+Test the build pipeline with:
+
+```bash
+act -s NPM_AUTH -s NPM_USER_EMAIL -W .github/workflows/ci.yml
+```
+
+See "Testing Pipelines" in the developer guide for how to setup `act`.