Geting directly stdout via $cmd
#1
Comments
|
First of all, thanks for the support :) I originally planed to explain why I designed I tried to focus Also, from what I saw, some
The main idea is to stay close to bash while taking advantage of JavaScript features. This test for instance shows what is possible when we bring the best of both worlds (it may even be possible to pipe a WebSocket connection into commands easily for instance - not really useful but it's impossible to do with bash or zx).
I don't want to surcharge the import { $, collect } from 'https://deno.land/x/bazx/mod.ts';
const $raw = async (...args: Parameters<typeof $>) => (await collect($(...args))).stdout;
const hello = await $raw`echo hello`;
Maybe you should take a look at the test folder, there are more example on how |
|
Thank you for detailed explanation! I understand what you want to resolve. The first thing I felt was that this writing style is a bit redundant. let branch = (await stdout($`git branch`)).stdoutWhat I was thinking was that it would be better to be able to write more frequently used functions in a concise manner. Even if all the standard output is loaded to memory at once, we may not encounter such a process that results in out of memory very often. (it depends) For easier scripting, it is easy to define my own function, such as This discussion is based on the situation I consider when writing shell scripts, so it may be quite different from what you assume. Thanks |
|
Ok, I see what you mean. I had to do things like let { code, stdout: branch, stderr: warnings } = await collect($`git branch`))But I agree it's not really efficient for the common use case. I thought about it and I realized the standard let branch = await $`git branch`.text()This way I can split the API in 2 parts:
function exec(cmd: [string, ...string[]], streams?: {
stdin: ReadableStream<Uint8Array>,
stdout: WritableStream<Uint8Array>,
stderr: WritableStream<Uint8Array>
}): Promise<{ code: number }>
let b = await $`echo ${foo}`.pipe($`tr f F`).text()I like this compromise since users that wants their own features don't need to import everything, while users that just wants to keep code simple and close to bash don't have to bring their own wrappers everywhere. What do you think? Does it fit your needs? |
|
Looks great! Thank you for your understanding. There are a few things I would like to confirm.
$ cat sample.ts
import { $, stdout } from 'https://deno.land/x/bazx/mod.ts';
await $`ls -1`.pipe($`grep hoge`);
$ deno run --allow-run sample.ts
$ ls -1 | grep hoge
error: Uncaught (in promise) Error: Command "ls -1 | grep hoge" exited with code 1
throw new Error(`Command "${this}" exited with code ${res.code}`);
^
at https://deno.land/x/[email protected]/src/bazx.ts:45:15
at async file:///.../sample.ts:3:1Thanks |
Actually the fact that non-0 status code throws is intended to be configurable, but configuration is not implemented yet 😅 But as you said, errors handling is complicated here since a boolean in the configuration would disable errors for every commands, whereas users may only want to disable errors for Another solution might be to let the user provide a predicate that check whether a specific command and status code should throw, like for instance: options.throwsIf = (code: number, command: string) => (command === 'grep' && code === 1 ? false : code !== 0)It's not really straightforward, but I don't find any other way to deal with this situation 😕
I would say by default yes. The standard error output is supposed to report issues or at least to provide visual information to the user that shouldn't be directly processed by a program. However some users may want to process stderr, but for now I think it's not the main use case, and I don't want to complicate the surface API much more. (I guess that's also what you meant when you said
I think it could be possible to support all of them (only properties and functions, not the whole specification about behaviors), even if some properties like However I haven't decided yet whether |
Sorry for the vague wording. I agree with you.
Regarding to error handling, I'll try to read bazx code and think about another solution.
There are not many cases where I want to handle stderr output, so I thought It's just an idea. It might be a bit tricky I think. interface Response {
text: () => string
ok: boolean
// ...
}
type CommandResponse = Response | {
// ...
stderr: Response;
} |
Nice 👍 Currently, errors are thrown here: Lines 42 to 48 in 25ded30 I couldn't find the time to add a custom Error subclass with command status info (like
That's an interesting idea ( What about: interface Command extends Response {
stdout: Body;
stderr: Body;
}Since Response implements Body, all Body-related method/getters in Command would be forwarded to the The only issue with interface Command extends Body, PromiseLike<{ ok: boolean, status: number }> {
cmd: [string, ...string[]];
stdout: Body;
stderr: Body;
}
// example
let cmd = $`echo Hello world!`
let [message, { status }] = await Promise.all([cmd.text(), cmd]) |
Wow! That is just what I wanted to say! (Thank you for reading my mind)
Exactly. However, for simple APIs that don't support streams, I think it's possible to get stdout/stderr and status at the same time. zx's ProcessOutput does exactly that, allowing us to get all the values at the same time.
Looks good to me. BTW, when I looked at the code for zx, I noticed that one of the differences between zx and bazx is the way the child processes are launched. Since zx runs processes through a shell by default, it is possible to use bash notation in As for me, it is better to have no dependency on bash (Getting stuck due to the problem of different versions of bash being used is very annoying), so the current form that directly starts the process is better. However, I'd still like to have some kind of simple error suppressing method. |
Even though I have read |
Not on purpose ^^' Maybe we just share the same goal ;)
Yes but it means collecting all commands output by default 😕 This gives me another idea: interface Command extends
Body, // redirected to stdout, methods throws if status != 0 or command executable isn't found
PromiseLike<Response> { // returns property `stdout`
/**
* Executes the command without collecting outputs,
* then returns its exit code
* @throws only if the command executable is not found
*/
status: Promise<number>;
/**
* Executes the command without collecting outputs,
* then returns whether the command exited successfully or not
* @throws only if the command executable is not found
*/
ok: Promise<boolean>;
/**
* Executes the command while collecting stdout,
* then returns a Response representing its exit code and collected data
* @throws only if the command executable is not found
*/
stdout: Promise<Response>;
/**
* Executes the command while collecting stderr,
* then returns a Response representing its exit code and collected data
* @throws only if the command executable is not found
*/
stderr: Promise<Response>;
/**
* Execute the command while collecting both stdout and stderr (like `2>&1`),
* then returns a Response representing its exit code and collected data
* @throws only if the command executable is not found
*/
combined: Promise<Response>;
/** Creates a new Command that pipes `this` command to the `command` command */
pipe(command: Command): Command;
}
// usage
let result: Response = await $`ls -1`.pipe($`grep hoge`); // collects data; do NOT throw if status != 0
if (result.ok) {
console.log('Found:', await result.text()); // do NOT throw even if `result.status` != 0
}
if (await $`ls -1`.pipe($`grep hoge`).ok) { // runs `ls -1 | grep hoge` WITHOUT collecting output; do NOT throw if status != 0
console.log('Found something');
}
const found = await $`ls -1`.pipe($`grep hoge`).text(); // throw if exit code is not 0
// reusable commands
let cmd = $`ls -1`.pipe($`grep hoge`); // do NOT spawn any process (it's like a command template)
await cmd.ok; // runs cmd (spawn processes and wait for them to exit)
await cmd.ok; // runs cmd again (processes are spawned again - it may return a difference value)This way it's easy to disable outputs collection (with the
Does the above proposal meet your needs?
It's also easier to deal with in code (no injection issues, no OS specific cases to deal with, and so on), and users that still want a shell can simply use something like
https://github.com/google/zx/blob/41834646c901ce9647ab14ee8b9ffe1c9a581270/index.mjs#L183-L221
Currently |
|
Awesome! The example you provided looks close to ideal interface. A let result: Response = await $`ls -1`.pipe($`grep hoge`); // collects data; do NOT throw if status != 0
if (result.ok) {
console.log('Found:', await result.text()); // do NOT throw even if `result.status` != 0
}B const found = await $`ls -1`.pipe($`grep hoge`).text(); // throw if exit code is not 0It is a little difficult to understand when an error is thrown (B) and when it is not (A).
Thank you, I missed that ProcessOutput itself extends Error. |
The idea is that it depends on where the A const found = await (await $`ls -1`.pipe($`grep hoge`)).text(); // do NOT throw (`.text()` called on Response)B const found = await $`ls -1`.pipe($`grep hoge`).text(); // do throw (`.text()` called on Command)I agree it can be a bit confusing but I think it's a good compromise between API simplicity and feature richness. Anyway, Response might not be a good idea finally: If I want to implement the Response API I'll have to override the |
Understood, thank you! I like this rule, so simple.
There may not be much need to be concerned about complying with the standard Response interface strictly. What we need are just |
I just wrote a proof-of-concept... It was simpler than expected 😄 EDIT: This poc is much more than EDIT 2: Finally I rewrote everything from scratch, the new implementation is much better https://github.com/Minigugus/bazx/tree/dev Lines 3 to 11 in 6f5475f Also, I took the opportunity to add middlewares, I think it's a great addition to |
* Add Response to the core (see #1) * Add middlewares supports * Add better examples * Drop old implementation
|
Perfect! I was surprised at how quickly you implemented it! |
|
(This might be another issue) reading line one by oneOpening file (or Piping another command) as stream and reading line by line are common use case, but Deno requires boring boilerplate codes. I believe that it is better to have a utility function that can do this, something like following import { readerFromStreamReader } from "https://deno.land/std/io/streams.ts";
import { BufReader } from "https://deno.land/std/io/bufio.ts";
import { TextProtoReader } from "https://deno.land/std/textproto/mod.ts";
const fetchRes = await fetch(
"https://raw.githubusercontent.com/mayankchoubey/deno-doze/main/doze.ts",
);
for await (const line of lineReaderFromStream(fetchRes.body!)) {
console.log(line);
}
function lineReaderFromStream(
stream: ReadableStream<Uint8Array>,
): AsyncIterable<string> {
const streamReader = readerFromStreamReader(stream.getReader());
const bufReader = new BufReader(streamReader);
const tp = new TextProtoReader(bufReader);
return {
async *[Symbol.asyncIterator]() {
while (true) {
const line = await tp.readLine();
if (line == null) {
break;
}
yield line;
}
},
};
}(EDIT 1) reading user input interactivelyIt would be better to provide simple function like zx's (EDIT 2) checking file conditionsChecking if a file exists via just Some checking conditions can be done by Deno std library. Should I manage to do all these cases using Deno directly? |
https://deno.land/[email protected]/io/bufio.ts#L701-L719 + https://deno.land/[email protected]/io/streams.ts#L74-L94 = import { readLines, readerFromStreamReader } from 'https://deno.land/[email protected]/io/mod.ts';
import $ from 'https://raw.githubusercontent.com/Minigugus/bazx/71767d812c7f759d3160bc7f6834c0330876e4b7/deno.ts';
const lines = [];
for await (const line of readLines(readerFromStreamReader(
$`curl ${'https://raw.githubusercontent.com/mayankchoubey/deno-doze/main/doze.ts'}`.body.getReader()
))) {
lines.push(line);
}
console.log(lines);(that's why I like Deno 🙂)
For theses reason, you should manage to do these case with the runtime you are using. However, nothing prevent you from writing your own package around |
|
Understood! So If I want to provide utility functions for me and other bazx users, I can just publish it in my own repository myself. |
It is a very interesting product! I also love to use Deno & TypeScript.
I generally agree with the choice to make the default behavior a stream. However, I think the reason why zx is recognized as an alternative to bash is because the behavior of $
cmdis close to that of bash as follows:Have you thought about supporting streams by providing options like $.raw
cmdor $.streamcmd?The text was updated successfully, but these errors were encountered: