Irregular objects selection

[wino45]

Hi

I am trying to select all objects within an array, but some objects does not have all attributes.
echo '{ "users": [ { "name": "Tom" }, { "wrong_name": "Jim" } ] }' | dasel -r json 'users.all().name'

I would expect that this simple query returns first array element, while it returns nothing. Can I filter based on existence of an attribute ?

[TomWright]

There is something strange going on with the error output that I've been meaning to look into, but this would be throwing an error when it see's an object without the name.

You can get this working by adding a ? after the "optional" property name, e.g:

echo '{ "users": [ { "name": "Tom" }, { "wrong_name": "Jim" } ] }' | dasel -r json 'users.all().name?'
"Tom"

This is documented here: https://daseldocs.tomwright.me/functions/property

If any argument ends with a ?, that lookup is considered optional and dasel will continue if that property is not found.

[wino45]

Yeah, lack of error is misleading (it was Dasel 2.2).
When reading the property documentation it is not obvious that using ? can be used as filter. Adding one example can help.

echo '{
  "users": [
    {
      "name": "Tom"
    },
    {
      "wrong_name": "Jim"
    }
  ]
}' | dasel -r json 'users.all().filter(name?)'
{
  "name": "Tom"
}

[TomWright]

The ? operator isn't necessarily used as a filter although I can see how it appears that way.
Rather the filter func is using the string it finds there and evaluating it as truthy, whilst also skipping any items that returned a not found error.

This is clearer in this example:

echo '{
  "users": [
    {
      "name": "Tom"
    },
    {
      "name": "false"
    },
    {
      "name": "true"
    },
    {
      "wrong_name": "Jim"
    }
  ]
}' | dasel -r json 'users.all().filter(name?)'
{
  "name": "Tom"
}
{
  "name": "true"
}

dasel/func_filter.go

Lines 16 to 31 in 7b3a782

	runComparison := func(value Value, selector string) (bool, error) {
	gotValues, err := c.subSelect(value, selector)
	if err != nil {
	return false, err
	}
	if len(gotValues) > 1 {
	return false, fmt.Errorf("filter expects selector to return a single value")
	}
	if len(gotValues) == 0 {
	return false, nil
	}
	return IsTruthy(gotValues[0]), nil
	}

[TomWright]

As a side note, I've just fixed the error display issue. Releasing v2.3.3 with this fix now.

$ echo '{ "users": [ { "name": "Tom" }, { "wrong_name": "Jim" } ] }' | dasel -r json 'users.all().name' 
Error: could not access map index: property not found: name
exit status 1

[wino45]

So the proper object selection is:

echo '{
  "users": [
    {
      "name": "Tom"
    },
    {
      "name": "false"
    },
    {
      "name": true
    },
    {
      "wrong_name": "Jim"
    }
  ]
}' | dasel -r json 'users.all().name?.parent()'
{
  "name": "Tom"
}
{
  "name": "false"
}
{
  "name": true
}

I would not find this without your help. Thank you.

[wino45]

Any idea how to reverse the search (find objects that does not have a property) ?

This almost work :

echo '{
  "users": [
    {
      "name": "Tom"
    },
    {
      "name": "false"
    },
    {
      "name": true
    },
    {
      "wrong_name": "Jim"
    }
  ]
}' | dasel -r json 'users.all().filter(not(name?))'
{
  "name": "false"
}
{
  "wrong_name": "Jim"
}

[TomWright]

Note that some of the following examples will only work with dasel v2.3.4 and up due to some bugs that have been fixed.

This shows you users that have a name property that is not blank.

$ echo '{
  "users": [
    {
      "name": "Tom"
    },
    {
      "name": "false"
    },
    {
      "name": true
    },
    {
      "wrong_name": "Jim"
    },
    {
      "name": ""
    }
  ]
}' | dasel -r json 'users.all().filter(name?.len())'
{
  "name": "Tom"
}
{
  "name": "false"
}
{
  "name": true
}

This shows you users that have no name field, or a blank name field.

$ echo '{
  "users": [
    {
      "name": "Tom"
    },
    {
      "name": "false"
    },
    {
      "name": true
    },
    {
      "wrong_name": "Jim"
    },
    {
      "name": ""
    }
  ]
}' | dasel -r json 'users.all().filter(not(name?.len()))'
{
  "wrong_name": "Jim"
}
{
  "name": ""
}

This shows you users that have a name field, regardless of whether it is blank or not.

$ echo '{
  "users": [
    {
      "name": "Tom"
    },
    {
      "name": "false"
    },
    {
      "name": true
    },
    {
      "wrong_name": "Jim"
    },
    {
      "name": ""
    }
  ]
}' | dasel -r json 'users.all().filter(keys().all().filter(equal(.,name)))'
{
  "name": "Tom"
}
{
  "name": "false"
}
{
  "name": true
}
{
  "name": ""
}

This shows you users that do NOT have a name field, regardless of whether it is blank or not.

$ echo '{
  "users": [
    {
      "name": "Tom"
    },
    {
      "name": "false"
    },
    {
      "name": true
    },
    {
      "wrong_name": "Jim"
    },
    {
      "name": ""
    }
  ]
}' | dasel -r json 'users.all().filter(not(keys().all().filter(equal(.,name)).len()))'
{
  "wrong_name": "Jim"
}

[wino45]

Why len() is needed in the last example? It works without it:

$ echo '{
  "users": [
    {
      "name": "Tom"
    },
    {
      "name": "false"
    },
    {
      "name": true
    },
    {
      "wrong_name": "Jim"
    },
    {
      "name": ""
    }
  ]
}' | dasel -r json 'users.all().filter(not(keys().all().filter(equal(.,name))))'
{
  "wrong_name": "Jim"
}

What about defining a new function "isKey(x)" = keys().all().filter(equal(.,x)), so the example will become:
dasel -r json 'users.all().filter(not(isKey(name)))'
I would be very easy to define any combination of logic based on existence of a key, example:
dasel -r json 'users.all().filter(not(or(isKey(name),isKey(wrong_name))))'

BTW. The Equal documentation page uses direct Go invocation in the example.