Rewrite and refactor library

* Add Response to the core (see #1) * Add middlewares supports * Add better examples * Drop old implementation
Minigugus · Jun 11, 2021 · 6f5475f · 6f5475f
1 parent 95c104e
commit 6f5475f
Show file tree

Hide file tree

Showing 18 changed files with 749 additions and 642 deletions.
diff --git a/README.md b/README.md
@@ -4,117 +4,120 @@
 > 🐚️ [zx](https://github.com/google/zx) on 💊️ steroids
 </center>
 
-[![asciicast](https://asciinema.org/a/ydfYbBXFyDDyDOeSPormkjEo6.svg)](https://asciinema.org/a/ydfYbBXFyDDyDOeSPormkjEo6)
+> You are seeing the code of an upcoming release. The `main` branch contains the latest released code.
+> Code on this branch is still under discussion and documentation is not completed yet.
 
 ## Main differences with ZX
 
  * Written from scratch
  * **0 dependencies** by default
  * **Plateform-agnostic** (not limited to *Deno* or *NodeJS*)
  * Extensible (enables **remote command execution** and thus **browser support**)
+ * **Middlewares** support (filter allowed commands, hack input/outputs, and much more)
  * [**Streams**](#streams) at the core for better performances ([no streaming support with zx](https://github.com/google/zx/issues/14#issuecomment-841672494))
- * Tree-shakable (usefull for **bundling**)
+ * No shell by default (the `|` in ``$`echo |`;`` is not evaluated by default)
  * Modern (TypeScript, Deno, ES2020)
  * **MIT** vs Apache license
  * Library only
 
-### Streams
-
-Unlike zx, bazx doesn't gather outputs into possibly big strings by default. The main reason for that is to avoid out of memory, especially on embedded devices and with "loud" commands (for instance, ``await $`yes`;`` with zx *will* crash the process, but not with bazx).
-
 ## Support
 
  * Deno: 🐥️ (working)
  * NodeJS: 🥚️ (not started yet)
  * QuickJS: 🥚️ (not started yet)
  * Browser: 🥚️ (not started yet)
- * Mock: 🥚️ (not started yet)
+ * Mock: 🐣️ (started)
 
 ## Setup
 
 ### Deno
 
 ```js
-import { $ } from 'https://deno.land/x/bazx/mod.ts';
+import $ from 'https://deno.land/x/bazx/deno.ts';
 ```
 
 As of now, only the `--allow-run` command is required at runtime.
 
+### Isomorphic (for testing purpose)
+
+```js
+import $ from 'https://deno.land/x/bazx/mock.ts';
+```
+
+This implementation doesn't know how to spawn a process and thus always throw.
+Intended to be used along with middlewares in tests or sandboxes for instance.
+
 ### (Bring Your Own Runtime)
 
 ```ts
-// `index.ts` is isomorphic, `mod.ts` is Deno only
-import { create } from 'https://deno.land/x/bazx/index.ts';
-
-const $ = create({
-  exec(
-    cmd: [string, ...string[]],
-    streams: {
-      stdin: ReadableStream<Uint8Array>,
-      stdout: WritableStream<Uint8Array>,
-      stderr: WritableStream<Uint8Array>
-    }
-  ): PromiseLike<{ success: boolean, code: number }> {
-    // Create thread here
+import { createBazx } from 'https://deno.land/x/bazx/mod.ts';
+
+const $ = createBazx((
+  cmd: [string, ...string[]],
+  options: {
+    cwd?: string,
+    env?: Record<string, string>,
+    stdin?: ReadableStream<Uint8Array>,
+    stdout?: WritableStream<Uint8Array>,
+    stderr?: WritableStream<Uint8Array>,
+    signal?: AbortSignal,
   }
-}, { /* Default options */ });
+): PromiseLike<{ code: number }> => {
+  // Spawn commands here.
+  //
+  // CAUTION: This function is reponsible for closing stdin/stdout/stderr.
+  //          Missing to do so will result in deadlocks.
+}, { /* Default options (logging, colors, and so on) */ });
 ```
 
 ## Usage
 
-See the [`test`](test/) folder for more complete examples
-
-### The `$` tagged template function
+See the [`examples`](examples/) folder for more examples
 
-Prepare a command. The process will spawn only when the returned object will be awaited (`then` called). The returned object as `stdin`, `stdout` and `stderr` properties, that are streams the user can use to communicated with the process. Also, it is possible to call the `pipe(cmdOrTransfertStream: NativeCommand | PipeCommand | TransformStream)` function to pipe the *this* command into the provided `cmdOrTransfertStream` stream, or to `stdin` stream of the provided command.
+### Middlewares
 
-```js
-$`echo Never called`; // (never executed - missing `await`)
+Middlewares are hooks that runs a process get spawned, so that it can for instance
+dynamically hack streams, change the command line, working directories and environment
+variabes, never really spawn a process, spawn a process twice, and so on.
 
-await $`echo Hi!`; // $ echo Hi!
+For instance, this really simple middleware wraps processes with `time`, so that some process meta are displayed:
 
-const cmd = $`echo Only invoked once`;
-(await cmd) === (await cmd); // => true
+```typescript
+import { BazxMiddleware } from 'https://deno.land/x/bazx/mod.ts';
 
-await $`env`.pipe($`grep PATH`); // $ env | grep PATH
+export const timeWrapperMiddleware: BazxMiddleware =
+  exec => (cmd, options) => exec(['time', ...cmd], options);
 ```
 
-The `$` function can be obtained using the `create` function, or from a third party module that wrap this function. For instance, the [`deno.ts`](deno.ts) module expose a `createDeno` function, which is a wrapper function around `create`.
-
-See [`test/debug.js`](test/debug.js) for a pratical overview.
+Then, it can be applied with the `$.with` function:
 
-### `stdout`, `stderr` and `collect` exports
+```typescript
+import $ from 'https://deno.land/x/bazx/deno.ts';
 
-Utilities to read respectively `stdout`, `stderr` and both from the command passed as argument:
-
-```js
-import { stdout } from 'https://deno.land/x/bazx/index.ts';
+const $$ = $.with(timeWrapperMiddleware);
 
-console.log(await stdout($`echo Hello world!`));
+await $$`echo Hello world!`.status
 
-// => { success: true, code: 0, stdout: "Hello world!" }
+// $ echo Hello world!
+// Hello world!
+// 0.00user 0.00system 0:00.00elapsed 100%CPU (0avgtext+0avgdata 1936maxresident)k
+// 0inputs+0outputs (0major+73minor)pagefaults 0swaps
 ```
 
-### Environment variables
-
-Environment variables are inherited from the runtime. For instance, with *Deno*, you must use `Deno.env` with the appropriate permission to change environment variables passed to the child process (`Deno.run` inherits `Deno.env` by default).
-
-Environment variables are still under discussion. Suggestions welcomed :slightly_smiling_face:.
-
-## WIP
-
-This project is a work in progress for now; bugs and API change are expected.
+As you can may have noticed, only the original command is printed to the user,
+not the updated one with `time`. This way, middlewares are fully transparents to the user.
 
-Please fill an issue for any bug you encounter and open a discussion for any question or suggestion. :wink:
+Each `.with` call returns a new `$` instance that use the config of the parent (`exec`, `options` and middlewares),
+so that multiple `.with` calls can be chained.
 
 ### TODO
 
- * [ ] Improve docs
+ * [ ] Improve docs (README + JSDoc)
  * [ ] Rollup for NodeJS and browser builds
  * [ ] Add more runtime support (NodeJS at least)
  * [ ] Fix bugs (some complex use case doesn't work yet)
- * [ ] Dynamic config update (like `set` in bash (enable/disable logs, etc.))
- * [ ] `NO_COLOR` support (for CI/CD)
+ * [X] Dynamic config update (like `set` in bash (enable/disable logs, etc.))
+ * [X] `NO_COLOR` support (for CI/CD)
  * [ ] Pipelines starting with a stream
  * [ ] `stderr` pipes
  * [ ] Add benchmarks, improve perfs (audit WHATWG streams perfs)

diff --git a/deno.ts b/deno.ts
@@ -1,62 +1,12 @@
-/// <reference path="./deno.d.ts" />
+// Bazx implementation for Deno.
 
-import { create, BazxOptions } from './mod.ts';
+/// <reference path="./deno.d.ts" />
 
-export const $ = createDeno();
+export * from './mod.ts'
 
-export function createDeno(options?: Partial<BazxOptions>) {
-  return create({
-    async exec(cmd, { stdin, stdout, stderr }) {
-      const proc = Deno.run({
-        cmd,
-        stdin: stdin ? 'piped' : 'null',
-        stdout: stdout ? 'piped' : 'null',
-        stderr: stderr ? 'piped' : 'null'
-      });
-      const procIn = stdin && stdin.pipeTo(streamCopyFromStream(proc.stdin!));
-      const procOut = stdout && streamCopyToStream(proc.stdout!).pipeTo(stdout);
-      const procErr = stderr && streamCopyToStream(proc.stderr!).pipeTo(stderr);
-      const result = await proc.status();
-      await Promise.all([
-        procIn,
-        procOut,
-        procErr
-      ])
-      proc.close();
-      return result;
-    },
-    get stdout() {
-      return streamCopyFromStream({ write: chunk => Deno.stdout.write(chunk) });
-    },
-    get stderr() {
-      const decoder = new TextDecoder();
-      const encoder = new TextEncoder();
-      return new WritableStream<Uint8Array>({
-        async write(chunk) {
-          await Deno.stderr.write(encoder.encode(`\x1B[31m${decoder.decode(chunk)}\x1B[0m`));
-        }
-      }, new ByteLengthQueuingStrategy({
-        highWaterMark: 16_640
-      }));
-    }
-  }, options);
-}
+import type { BazxExec, BazxOptions } from './mod.ts';
 
-function streamCopyFromStream(writer: Deno.Writer & Partial<Deno.Closer>) {
-  return new WritableStream<Uint8Array>({
-    async write(chunk) {
-      await writer.write(chunk);
-    },
-    close() {
-      writer.close?.();
-    },
-    abort() {
-      writer.close?.();
-    }
-  }, new ByteLengthQueuingStrategy({
-    highWaterMark: 16_640
-  }));
-}
+import { createBaxz } from './mod.ts';
 
 function streamCopyToStream(reader: Deno.Reader & Deno.Closer) {
   const buffer = new Uint8Array(16_640);
@@ -66,7 +16,7 @@ function streamCopyToStream(reader: Deno.Reader & Deno.Closer) {
       try {
         while (controller.desiredSize! > 0) {
           if ((read = await reader.read(buffer.subarray(0, Math.min(buffer.byteLength, controller.desiredSize ?? Number.MAX_VALUE)))) === null) {
-            reader.close?.();
+            reader.close();
             controller.close();
             return;
           }
@@ -80,9 +30,103 @@ function streamCopyToStream(reader: Deno.Reader & Deno.Closer) {
       }
     },
     cancel() {
-      reader.close?.();
+      reader.close();
     }
   }, new ByteLengthQueuingStrategy({
-    highWaterMark: 16_640
+    highWaterMark: 16640
   }));
 }
+
+async function pipeReadableStream2Writer(
+  readable: ReadableStream<Uint8Array>,
+  writer: Deno.Writer & Deno.Closer
+) {
+  const reader = readable.getReader();
+  try {
+    let read: ReadableStreamReadResult<Uint8Array>;
+    while (!(read = await reader.read()).done)
+      if (!await writer.write(read.value!))
+        break;
+    await reader.cancel();
+  } catch (err) {
+    if (err instanceof Deno.errors.BrokenPipe)
+      await reader.releaseLock();
+    else
+      await reader.cancel(err);
+  } finally {
+    try {
+      writer.close();
+    } catch (ignored) { }
+  }
+}
+
+export const exec: BazxExec = async function exec(cmd, {
+  cwd,
+  env,
+  stdin,
+  stdout,
+  stderr,
+  signal
+} = {}) {
+  const process = Deno.run({
+    cmd,
+    cwd,
+    env,
+    stdin: stdin ? 'piped' : 'null',
+    stdout: stdout ? 'piped' : 'null',
+    stderr: stderr ? 'piped' : 'null',
+  });
+  signal?.addEventListener('abort', () => process.kill?.(9), { once: true });
+  try {
+    const [{ code, signal: exitSignal }] = await Promise.all([
+      process.status(),
+      stdin && pipeReadableStream2Writer(stdin, process.stdin!),
+      stdout && streamCopyToStream(process.stdout!).pipeTo(stdout),
+      stderr && streamCopyToStream(process.stderr!).pipeTo(stderr),
+    ]);
+    return { code, signal: exitSignal };
+  } finally {
+    process.close();
+  }
+}
+
+export const options: BazxOptions = {
+  highWaterMark: 16640,
+  noColors: Deno.noColor,
+  log: chunk => Deno.stdout.writeSync(chunk)
+};
+
+export const $ = createBaxz(exec, options);
+
+export default $;
+
+Deno.test({
+  name: 'everything works',
+  async fn() {
+    // @ts-ignore
+    const assert: (expr: unknown, msg: string) => asserts expr = (await import("https://deno.land/std/testing/asserts.ts")).assert;
+    // @ts-ignore
+    const { fail, assertEquals } = await import("https://deno.land/std/testing/asserts.ts");
+
+    const cmd = $`bash -c ${'echo Hello world! $(env | grep WTF) $(pwd)'}`
+      .cwd('/bin')
+      .pipe(new TransformStream({
+        start(controller) {
+          controller.enqueue(new TextEncoder().encode('Someone said: '));
+        }
+      }))
+      .env('WTF', 'it works')
+      .pipe($`sh -c ${'cat - $(realpath $(env | grep WTF))'}`)
+      .env('WTF', 'not_found')
+      .cwd('/');
+
+    try {
+      await cmd.text();
+      fail("Should have thrown since cat should have failed");
+    } catch (err) {
+      assertEquals(`Command ${cmd.command} exited with code 1`, err.message);
+      assert(err.response instanceof Response, "err.response is defined and is an instance of Response");
+      assertEquals('Someone said: Hello world! WTF=it works /bin\n', await err.response.text());
+    }
+  }
+});