feat: Assertion verification #452

elizabethhealy · 2024-12-09T02:25:40Z

Adds option to provide assertion verification keys and support for signing keys on assertion input
(supports rs256 and hs256 keys)
Includes bats tests for both key types

jentfoo

Specifying the keys directly as arguments puts the key at more risk of exposure. Arguments will be recorded to disk in the command history, could be exposed on multi-tenant systems, logged, etc. This is particularly concerning since you appear to be supporting not just public keys, but private keys as well.

@elizabethhealy Would it be reasonable to accept a file path or use environment variables instead of specifying the keys as arguments?

pkg/handlers/tdf.go

jakedoublev · 2024-12-10T16:17:08Z

Specifying the keys directly as arguments puts the key at more risk of exposure. Arguments will be recorded to disk in the command history, could be exposed on multi-tenant systems, logged, etc. This is particularly concerning since you appear to be supporting not just public keys, but private keys as well.

@elizabethhealy Would it be reasonable to accept a file path or use environment variables instead of specifying the keys as arguments?

This suggestion from @jentfoo aligns with an open issue we have to generally improve CLI security: #417

elizabethhealy · 2024-12-10T16:35:56Z

@jentfoo @jakedoublev
using a file name as input instead makes sense, so in that case the format of the assertions provided would be [{"id":"assertion1","type":"handling","scope":"tdo","appliesToState":"encrypted","statement":{"format":"json+stanag5636","schema":"urn:nato:stanag:5636:A:1:elements:json","value":"{\"ocl\":\"2024-10-21T20:47:36Z\"}"},"signingKey":{"alg":"RS256","key":"my_private_rs256key.pem"}}] and just require that whats in the key field is a path to a file
or should we support an additional field within "signingKey" like "key_file" where a user would pass the pem?

jakedoublev · 2024-12-10T16:42:20Z

@jentfoo @jakedoublev using a file name as input instead makes sense, so in that case the format of the assertions provided would be [{"id":"assertion1","type":"handling","scope":"tdo","appliesToState":"encrypted","statement":{"format":"json+stanag5636","schema":"urn:nato:stanag:5636:A:1:elements:json","value":"{\"ocl\":\"2024-10-21T20:47:36Z\"}"},"signingKey":{"alg":"RS256","key":"my_private_rs256key.pem"}}]

@elizabethhealy Could the entire assertion JSON be stored and read into the command from a file instead of just the keys referenced by the assertions? I'm thinking that would be aligned with what the CLI does for client credentials so --with-client-creds-file creds.json expects the entire {"clientId":"id","clientSecret":"client_secret"} stored in ./creds.json instead of --with-client-creds-file '{"clientId":"id","clientSecret":"secret_file.txt"}' where just the secret is present in ./secret_file.txt.

elizabethhealy · 2024-12-10T17:12:51Z

@jakedoublev i like that, so --with-assertions, --with-assertions-from-file, --with-assertion-verification-keys, --with-assertion-verificiation-keys-from-file and its up to the user to decide whichever is best for them, it will still support passing in keys directly but also give a safer option

jakedoublev · 2024-12-10T17:31:31Z

@elizabethhealy and I discussed and we will update --with-assertions to parse either from JSON (backwards-compatible) or try to read from a file (more secure) before reporting an error if neither is successful. This new decrypt flag --with-assertion-verification-keys will only read the entire JSON (including in-lined PEMs) from a file and not support JSON in the flag value/argument.

docs/man/decrypt/_index.md

docs/man/encrypt/_index.md

pkg/handlers/tdf.go

jakedoublev · 2024-12-10T20:39:45Z

pkg/handlers/tdf.go

+	return bytes, nil
+}
+
+func correctKeyType(assertionKey sdk.AssertionKey, public bool) (interface{}, error) {


Do you think this functionality could be exposed within provided platform lib/SDK functionality instead of recreated within each Go PEP?

seems like most go packages that deal with crypto just take in args that are already typed rsa.PrivateKey or rsa.PublicKey, they assume you already have the key and if not they expect you to load it yourself. i think the cli has a bit extra code since the whole assertion object is being passed in as a string which isnt how it would normally be done in a go pep. i assume the pep would just construct the assertion similar to https://github.com/opentdf/platform/blob/7d55a909ac03e5a1234709d9051f906d9efd469e/sdk/tdf_test.go#L400-L425 and would just load the key from pem; also the cli has extra conditions to handle private vs public where as i assume a pep would know if it was a private key or public key they were loading. so TLDR i think the cli is more of an outlier in how its passing in assertions and the extra steps needed to parse them and want to avoid adding specific logic to the sdk just for the cli.
but if its determined that other peps have this need as well we can re-assess

jentfoo · 2024-12-11T15:55:24Z

docs/man/encrypt/_index.md

@@ -27,7 +27,7 @@ command:
      default: /kas
    - name: with-assertions
      description: >
-        EXPERIMENTAL: JSON string of assertions to bind metadata to the TDF. See examples for more information.
+        EXPERIMENTAL: JSON string or path to JSON file of assertions to bind metadata to the TDF. See examples for more information.


I think we should only support the file method. A user may not be aware of the risks of specifying the key as a command argument.

We could consider adding argument support after #417 is implemented.

i think keeping the json string options was just to maintain backward compatability, we can add some more documentation here to emphasize that its deprecated and the file path is the more safe option

pkg/handlers/tdf.go

pkg/utils/read.go

jakedoublev · 2024-12-13T17:54:43Z

e2e/encrypt-decrypt.bats

@@ -57,6 +81,18 @@ teardown_file(){
  ./otdfctl decrypt --host $HOST --tls-no-verify $DEBUG_LEVEL $WITH_CREDS $OUTFILE_TXT | grep "$SECRET_TEXT"
 }

+@test "roundtrip TDF3, assertions with HS265 keys and verificaion, file" {


Suggested change

@test "roundtrip TDF3, assertions with HS265 keys and verificaion, file" {

@test "roundtrip TDF3, assertions with HS265 keys and verification, file" {

jakedoublev · 2024-12-13T17:54:51Z

e2e/encrypt-decrypt.bats

+  diff $INFILE_GO_MOD $RESULTFILE_GO_MOD
+}
+
+@test "roundtrip TDF3, assertions with RS256 keys and verificaion, file" {


Suggested change

@test "roundtrip TDF3, assertions with RS256 keys and verificaion, file" {

@test "roundtrip TDF3, assertions with RS256 keys and verification, file" {

jakedoublev · 2024-12-13T17:55:40Z

e2e/encrypt-decrypt.bats

+}
+
+@test "roundtrip TDF3, assertions with RS256 keys and verificaion, file" {
+  ./otdfctl encrypt -o $OUTFILE_GO_MOD --host $HOST --tls-no-verify $DEBUG_LEVEL $WITH_CREDS -a $FQN --with-assertions $SIGNED_ASSERTIONS_RS256 --tdf-type tdf3 $INFILE_GO_MOD


Are assertions present in the manifest? If so, can we add a test with ./otdfctl inspect that asserts they were found present in the TDF created?

jakedoublev · 2024-12-13T17:58:40Z

pkg/handlers/tdf.go

+				}
+			}
+			for i, config := range assertionConfigs {
+				if (config.SigningKey != sdk.AssertionKey{}) {


Does this comparison struct equality evaluate as expected? It seems like you'd need to compare struct fields for this equality check?

elizabethhealy added 11 commits December 8, 2024 21:24

assertion verification

6de7a5f

bats more error logging

33a806e

add rs256 support

5eacf3c

linting, try to do bats differently

ce33b1e

update to bats 3.0

cd422cd

no -f

a4fa180

load asserts correctly

5e812be

use output var

fffb2ef

use file instead of stdin

898d500

switch back to main

33de674

support case where there is no signing key

2db8d06

elizabethhealy marked this pull request as ready for review December 10, 2024 15:00

elizabethhealy requested a review from a team as a code owner December 10, 2024 15:00

jentfoo reviewed Dec 10, 2024

View reviewed changes

pkg/handlers/tdf.go Outdated Show resolved Hide resolved

elizabethhealy added 5 commits December 10, 2024 13:26

take json file instead of string

4a90670

add exports, remove test files on finish

45bea1e

linting

ae23aa8

linting, better function signature

2ff3f96

remove accidentally committed test files

b77b3b8

elizabethhealy requested review from jentfoo and jakedoublev December 10, 2024 19:13

jakedoublev reviewed Dec 10, 2024

View reviewed changes

implement suggestions

73baae7

jentfoo reviewed Dec 11, 2024

View reviewed changes

fixed read sizes, docs improvements, cleanup

a2dd130

spellcheck

19523aa

jakedoublev reviewed Dec 13, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Assertion verification #452

feat: Assertion verification #452

elizabethhealy commented Dec 9, 2024 •

edited

Loading

jentfoo left a comment

jakedoublev commented Dec 10, 2024

elizabethhealy commented Dec 10, 2024 •

edited

Loading

jakedoublev commented Dec 10, 2024

elizabethhealy commented Dec 10, 2024 •

edited

Loading

jakedoublev commented Dec 10, 2024

jakedoublev Dec 10, 2024

elizabethhealy Dec 13, 2024 •

edited

Loading

jentfoo Dec 11, 2024

elizabethhealy Dec 11, 2024 •

edited

Loading

jakedoublev Dec 13, 2024

jakedoublev Dec 13, 2024

jakedoublev Dec 13, 2024

jakedoublev Dec 13, 2024

	@test "roundtrip TDF3, assertions with HS265 keys and verificaion, file" {
	@test "roundtrip TDF3, assertions with HS265 keys and verification, file" {

	@test "roundtrip TDF3, assertions with RS256 keys and verificaion, file" {
	@test "roundtrip TDF3, assertions with RS256 keys and verification, file" {

feat: Assertion verification #452

Are you sure you want to change the base?

feat: Assertion verification #452

Conversation

elizabethhealy commented Dec 9, 2024 • edited Loading

jentfoo left a comment

Choose a reason for hiding this comment

jakedoublev commented Dec 10, 2024

elizabethhealy commented Dec 10, 2024 • edited Loading

jakedoublev commented Dec 10, 2024

elizabethhealy commented Dec 10, 2024 • edited Loading

jakedoublev commented Dec 10, 2024

jakedoublev Dec 10, 2024

Choose a reason for hiding this comment

elizabethhealy Dec 13, 2024 • edited Loading

Choose a reason for hiding this comment

jentfoo Dec 11, 2024

Choose a reason for hiding this comment

elizabethhealy Dec 11, 2024 • edited Loading

Choose a reason for hiding this comment

jakedoublev Dec 13, 2024

Choose a reason for hiding this comment

jakedoublev Dec 13, 2024

Choose a reason for hiding this comment

jakedoublev Dec 13, 2024

Choose a reason for hiding this comment

jakedoublev Dec 13, 2024

Choose a reason for hiding this comment

elizabethhealy commented Dec 9, 2024 •

edited

Loading

elizabethhealy commented Dec 10, 2024 •

edited

Loading

elizabethhealy commented Dec 10, 2024 •

edited

Loading

elizabethhealy Dec 13, 2024 •

edited

Loading

elizabethhealy Dec 11, 2024 •

edited

Loading