Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CSV stringification functionality and remove verbose error logging #726

Merged
merged 3 commits into from
Sep 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 35 additions & 10 deletions docs/genaisrc/genaiscript.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

45 changes: 35 additions & 10 deletions genaisrc/genaiscript.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

45 changes: 35 additions & 10 deletions packages/auto/genaiscript.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions packages/core/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
"ajv": "^8.17.1",
"cross-fetch": "^4.0.0",
"csv-parse": "^5.5.6",
"csv-stringify": "^6.5.1",
"dotenv": "^16.4.5",
"esbuild": "^0.24.0",
"fast-xml-parser": "^4.5.0",
Expand Down
32 changes: 31 additions & 1 deletion packages/core/src/csv.test.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { describe, test, beforeEach } from "node:test"
import assert from "node:assert/strict"
import { CSVParse, CSVTryParse, CSVToMarkdown } from "./csv"
import { CSVParse, CSVTryParse, CSVToMarkdown, CSVStringify } from "./csv"

describe("CSVParse", () => {
test("Parse simple CSV data with default options", () => {
Expand Down Expand Up @@ -82,3 +82,33 @@ describe("CSVToMarkdown", () => {
assert.equal(result, "")
})
})
describe("CSVStringify", () => {
test("Stringify simple CSV data with default options", () => {
const csv = [
{ name: "John", age: "30" },
{ name: "Jane", age: "25" },
]
const result = CSVStringify(csv)
const expected = "John,30\nJane,25\n"
assert.equal(result, expected)
})
test("Stringify simple CSV data with headers", () => {
const csv = [
{ name: "John", age: "30" },
{ name: "Jane", age: "25" },
]
const result = CSVStringify(csv, { header: true })
const expected = "name,age\nJohn,30\nJane,25\n"
assert.equal(result, expected)
})

test("Stringify CSV data with custom delimiter", () => {
const csv = [
{ name: "John", age: "30" },
{ name: "Jane", age: "25" },
]
const result = CSVStringify(csv, { header: true, delimiter: "|" })
const expected = "name|age\nJohn|30\nJane|25\n"
assert.equal(result, expected)
})
})
30 changes: 22 additions & 8 deletions packages/core/src/csv.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
// Import the CSV parsing function from the csv-parse library
// This module provides functions for parsing and converting CSV data,
// including error handling and conversion to Markdown table format.

import { parse } from "csv-parse/sync"
// Import the TraceOptions interface for logging trace information
import { TraceOptions } from "./trace"
import { stringify } from "csv-stringify/sync"

/**
* Parses a CSV string into an array of objects.
Expand Down Expand Up @@ -61,6 +63,18 @@
}
}

/**
* Converts an array of objects into a CSV string.
*
* @param csv - An array of objects to be converted into CSV format.
* @param options - Optional configuration for CSV stringification.
* @returns A string representing the CSV formatted data.
*/
export function CSVStringify(csv: object[], options?: CSVStringifyOptions) {
// Convert objects to CSV string using the provided options
return stringify(csv, options)
}

Check failure on line 76 in packages/core/src/csv.ts

View workflow job for this annotation

GitHub Actions / build

The `CSVStringify` function does not handle potential errors that may occur during the stringification process. Consider adding a try-catch block to handle any exceptions that may be thrown by the `stringify` function. 🛠️

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CSVStringify function does not handle potential errors that may occur during the stringification process. It's recommended to wrap the stringify function call in a try-catch block to handle any exceptions and provide a more graceful error handling.

generated by pr-review-commit missing_error_handling


/**
* Converts an array of objects into a Markdown table format.
*
Expand All @@ -70,7 +84,7 @@
* @returns A string representing the CSV data in Markdown table format.
*/
export function CSVToMarkdown(csv: object[], options?: { headers?: string[] }) {
if (!csv?.length) return ""
if (!csv?.length) return "" // Return empty string if CSV is empty

const { headers = Object.keys(csv[0]) } = options || {}
const res: string[] = [
Expand All @@ -84,11 +98,11 @@
const s = v === undefined || v === null ? "" : String(v)
// Escape special Markdown characters and format cell content
return s
.replace(/\s+$/, "")
.replace(/[\\`*_{}[\]()#+\-.!]/g, (m) => "\\" + m)
.replace(/</g, "lt;")
.replace(/>/g, "gt;")
.replace(/\r?\n/g, "<br>")
.replace(/\s+$/, "") // Trim trailing whitespace
.replace(/[\\`*_{}[\]()#+\-.!]/g, (m) => "\\" + m) // Escape special characters
.replace(/</g, "lt;") // Replace '<' with its HTML entity
.replace(/>/g, "gt;") // Replace '>' with its HTML entity
.replace(/\r?\n/g, "<br>") // Replace newlines with <br>

Check failure on line 105 in packages/core/src/csv.ts

View workflow job for this annotation

GitHub Actions / build

The replacement of '<' and '>' with 'lt;' and 'gt;' respectively is incorrect. It should be replaced with their correct HTML entities '&lt;' and '&gt;'. This could lead to incorrect data representation. 🚀

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The escape handling in the CSVToMarkdown function could lead to incorrect data representation. The '<' and '>' characters are replaced with 'lt;' and 'gt;', which are not valid HTML entities. They should be replaced with '<' and '>' respectively.

generated by pr-review-commit improper_escape_handling

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The replacement of '<' and '>' with 'lt;' and 'gt;' respectively is incorrect. It should be replaced with their correct HTML entities '<' and '>'. This could lead to incorrect data representation. 🚀

generated by pr-review-commit html_entity_escape

})
.join("|")}|` // Join columns with '|'
),
Expand Down
45 changes: 35 additions & 10 deletions packages/core/src/genaisrc/genaiscript.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion packages/core/src/globals.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// Import various parsing and stringifying utilities
import { YAMLParse, YAMLStringify } from "./yaml"
import { CSVParse, CSVToMarkdown } from "./csv"
import { CSVParse, CSVToMarkdown, CSVStringify } from "./csv"
import { INIParse, INIStringify } from "./ini"
import { XMLParse } from "./xml"
import {
Expand Down Expand Up @@ -46,6 +46,7 @@ export function installGlobals() {
// Freeze CSV utilities
glb.CSV = Object.freeze<CSV>({
parse: CSVParse,
stringify: CSVStringify,
markdownify: CSVToMarkdown,
})

Expand Down
Loading