CSV has multiple header lines when converting from ndjson

[kbrown]

input:

{
    "_id" : "6001c87baff07b000760f696",
    "title" : "Acme Demo Company Risk Review",
    "description" : "Acme Demo",
}

{
    "_id" : "6005a424aff07b0007619253",
    "title" : "Acme Industries",
    "description" : "Demo Quick",
}

command:

cat qq.json | dasel -r json -w csv 

output

_id,description,title
5cdee194f0f3640774b0b077,Acme Demo,Acme Demo Company
_id,description,title
5cdee194f0f3640774b0b077,Demo Quick,Acme Industries

[0x7FFFFFFFFFFFFFFF]

It's not valid json syntax. I guess you have to change it to valid json file before feeding to dasel. The syntax of your data is very similar to hjson. In fact, you just need to put them in a pair of [ and ] to make them valid hjson. Then you can easily convert to json syntax.

For example, I put your data in a file qq2.json.

C:\Users\user1\Desktop\test1>cat qq2.json
{
    "_id" : "6001c87baff07b000760f696",
    "title" : "Acme Demo Company Risk Review",
    "description" : "Acme Demo",
}
{
    "_id" : "6005a424aff07b0007619253",
    "title" : "Acme Industries",
    "description" : "Demo Quick",
}

C:\Users\user1\Desktop\test1>

Put them in a pair of square brackets:

C:\Users\user1\Desktop\test1>cat qq2.json | awk 'BEGIN {print"["}; {print $0}; END {print "]"}' RS="{]"
[
{
    "_id" : "6001c87baff07b000760f696",
    "title" : "Acme Demo Company Risk Review",
    "description" : "Acme Demo",
}
{
    "_id" : "6005a424aff07b0007619253",
    "title" : "Acme Industries",
    "description" : "Demo Quick",
}

]

C:\Users\user1\Desktop\test1>

Convert to valid json.

C:\Users\user1\Desktop\test1>cat qq2.json | awk 'BEGIN {print"["}; {print $0}; END {print "]"}' RS="{]" | hjson -j
[
  {
    "_id": "6001c87baff07b000760f696",
    "description": "Acme Demo",
    "title": "Acme Demo Company Risk Review"
  },
  {
    "_id": "6005a424aff07b0007619253",
    "description": "Demo Quick",
    "title": "Acme Industries"
  }
]

C:\Users\user1\Desktop\test1>

And process it with dasel

C:\Users\user1\Desktop\test1>cat qq2.json | awk 'BEGIN {print"["}; {print $0}; END {print "]"}' RS="{]" | hjson -j | dasel -r json -w csv
_id,description,title
6001c87baff07b000760f696,Acme Demo,Acme Demo Company Risk Review
6005a424aff07b0007619253,Demo Quick,Acme Industries

C:\Users\user1\Desktop\test1>

[kbrown]

My data (from MongoDB) Is ndjson (newline delimited json aka json lines). Yes, I could use awk (thank you) or jq (as jq supports ndjson) to put in an array first. But, as dasel wants to be "One tool to rule them all" it needs to support this format.

[TomWright]

It does indeed support this function: https://daseldocs.tomwright.me/usage/flags/merge-input-documents

[TomWright]

Dasel treats a newline in JSON as a separate document, similar to the --- separate in YAML.

You need to use the --merge-input-documents flag.

Example:

$ echo '{
    "_id" : "6001c87baff07b000760f696",
    "title" : "Acme Demo Company Risk Review",
    "description" : "Acme Demo"
}
{
    "_id" : "6005a424aff07b0007619253",
    "title" : "Acme Industries",
    "description" : "Demo Quick"
}' | dasel -r json -w csv --merge-input-documents
_id,description,title
6001c87baff07b000760f696,Acme Demo,Acme Demo Company Risk Review
6005a424aff07b0007619253,Demo Quick,Acme Industries

[kbrown]

Thank you @TomWright