Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

It would be useful to be able to access the line json on each line emit #77

Open
oliverfoster opened this issue Aug 19, 2024 · 2 comments

Comments

@oliverfoster
Copy link

oliverfoster commented Aug 19, 2024

processedData.forEach((row) => {
const line = this.processRow(row);
if (line === undefined) return;
this.onLine(line);
this.onData(this._hasWritten ? this.opts.eol + line : line);
this._hasWritten = true;
});

    processedData.forEach((row) => {
      const line = this.processRow(row);
      if (line === undefined) return;
      this.onLine(line, row); // <- add row here
      this.onData(this._hasWritten ? this.opts.eol + line : line);
      this._hasWritten = true;
    });

this.streamParser.onLine = (line) => this.emit('line', line);

this.streamParser.onLine = (line, row) => this.emit('line', line, row); 
@juanjoDiaz
Copy link
Owner

Hi @oliverfoster ,

Can you elaborate on why do you think that the unprocessed data is needed?
It sounds like you are trying to troubleshoot something?

@oliverfoster
Copy link
Author

oliverfoster commented Nov 11, 2024

I think I was trying to do two things independently, measure and display the conversion progress and also then separately to create indexes for the CSV to speed up subsequent processing. Both things are helped by being able to identify the source JSON of the given output line (its date, id, index etc).

I think I may have also used it for filtering subsets whilst converting. Such that lines containing various properties would end up in segregated csvs.

As it would be an optional second argument, it shouldn't cost much to pass along the raw JSON object reference I would think, but it can be useful for a variety of use-cases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants