Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Experimental subcommand-based interface #185

Open
wants to merge 62 commits into
base: main
Choose a base branch
from
Open

Conversation

ok-nick
Copy link
Contributor

@ok-nick ok-nick commented Jun 6, 2024

Changes in this pull request

Rework the CLI to use subcommands rather than optional arguments.

  • Introduces subcommands as follows
    • c2patool sign - sign manifest
    • c2patool extract - extract data
      • c2patool extract resources - extract resources
      • c2patool extract ingredient - extract ingredient json and binary manifest
      • c2patool extract manifest - extract json manifest (can extract .c2pa manifest with --binary )
    • c2patool view - view info about manifest
      • c2patool view manifest - view user-friendly json manifest (can specify --debug)
      • c2patool view ingredient - view json ingredient
      • c2patool view info - view general info about a manifest
      • c2patool view tree - view manifest tree
      • c2patool view certs - view manifest certs
  • More intuitive
    • With the old one, it's valid to specify c2patool i.jpg --info --certs --tree --detailed --manifest manifest.json --output o.jpg trust, but what's actually happening?
    • With the new one, it's impossible to make an ambiguous call, for each subcommand you can only specify the arguments relevant to that subcommand (otherwise clap will output a user-friendly error w/ suggestions)
    • Because of this, --help menus are much shorter and navigable, describing what subcommands/args are valid to specify
    • Subcommands also follow a familiar interface, similar to existing tools such as cargo, rustup, cross, git, etc.
  • Easier to modularize
    • Subcommands can be broken down into logical units, making it significantly easier to add new subcommands and modify existing ones
    • Subcommands can be constructed individually and tested individually
  • Migrated to the (new) c2pa-rs unstable API

Additional Changes

  • Added insta-rs snapshot integration tests (asserts outputs against a reference value)
  • Added --binary flag to extract manifest as .c2pa binary manifest
  • Added --no-verify flag to allow extracting invalid binary manifests for debugging
  • Added --unknown flag to additionally extract unstandardized labeled resources
  • Added --no-embed flag when signing to not embed a manifest in the asset.
  • Added glob as input path to sign multiple assets w/ the same manifest
  • Added --verbose flag to specify verbosity via flags (rather than only env vars)
  • Changed args to kebab-case (default) rather than a mix of kebab-case and snake_case
  • Changed --no-verify-signing to --no-verify
  • Changed check to allow different file extensions for input/output if --sidecar is specified
  • Changed ingredient_paths to be merged into ingredients field, specifying path or inline ingredient in .json
  • Changed --remote to --manifest-url and applied the same suffix-style pattern to trust args
  • Removed extracting ingredient .json also extracts .c2pa binary manifest (use extract manifest --binary)
  • Removed --remote in favor of specifying path or url in --manifest (similar to trust args)

Uncertain Changes

  • Removed checking for child ingredient.json if directory specified as ingredient path
  • Removed passing manifest as string via --config
  • Removed displaying manifest after signing (what would happen if multiple files are signed?)

Examples

# old
c2patool some_dir/C.jpg
# new - exactly the same
c2patool some_dir/C.jpg
# new - the above command redirects to below
c2patool view manifest some_dir/C.jpg
# new - let's view a detailed (debug) manifest
c2patool view manifest some_dir/C.jpg --debug

# old - let's view the certs
c2patool some_dir/C.jpg --certs
# new - the view subcommand contains: manifest, ingredient, info, tree, certs
c2patool view certs some_dir/C.jpg

# old - let's sign a manifest
c2patool some_dir/C.jpg --manifest some_dir/ingredient_test.json --output some_dir/C-signed.jpg
# new - very similar, it's now locked behind a subcommand
c2patool sign some_dir/C.jpg --manifest some_dir/ingredient_test.json --output some_dir/C-signed.jpg
# new - let's sign jpg files recursively
c2patool sign some_dir/**/*.jpg --manifest some_dir/ingredient_test.json --output some_dir/

# old - let's extract resources
c2patool some_dir/C.jpg --output some_dir
# new - now our intention is clear
c2patool extract resources some_dir/C.jpg --output some_dir
# new - I want to extract resources for all jpg files in some_dir
c2patool extract resources some_dir/*.jpg --output some_dir

Screenshots

c2patool
c2patool extract
c2patool view
c2patool view manifest -h
c2patool sign -h

Checklist

  • This PR represents a single feature, fix, or change.
  • All applicable changes have been documented.
  • Any TO DO items (or similar) have been entered as GitHub issues and the link to that issue has been included in a comment.

@codecov-commenter
Copy link

codecov-commenter commented Jun 6, 2024

Codecov Report

Attention: Patch coverage is 68.40077% with 164 lines in your changes missing coverage. Please review.

Project coverage is 74.43%. Comparing base (f216d3e) to head (e4e0553).
Report is 1 commits behind head on main.

Files Patch % Lines
src/commands/sign.rs 66.84% 63 Missing ⚠️
src/commands/extract.rs 60.37% 42 Missing ⚠️
src/commands/view.rs 63.72% 37 Missing ⚠️
src/main.rs 62.50% 15 Missing ⚠️
src/commands/mod.rs 88.13% 7 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@             Coverage Diff             @@
##             main     #185       +/-   ##
===========================================
+ Coverage   47.89%   74.43%   +26.53%     
===========================================
  Files           4        7        +3     
  Lines         666      841      +175     
===========================================
+ Hits          319      626      +307     
+ Misses        347      215      -132     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mauricefisher64
Copy link
Collaborator

Nice job

@ok-nick ok-nick added the enhancement New feature or request label Jun 7, 2024
Copy link
Contributor

@dkozma dkozma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR info looks amazing! Good work @ok-nick ! Will take a closer look at the code later today.

Copy link
Collaborator

@dyro dyro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you thought about how we'd roll this change out? Is the subcommand API the 1.0 version of c2patool?

#[clap(flatten)]
trust: Trust,
//
// TODO: add flag for additionally exporting unknown ingredients (ingredients that
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, this will be insanely useful.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the unstable API we should be able to support this functionality. Separate PR coming soon.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added: d67b6dc

trust,
} => {
if !path.exists() {
bail!("Input path does not exist")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about having distinct exit codes per error that the command encounters? We have another CLI tool for internal clients named adobe_c2patool that uses a similar subcommand model. In that tool, we have distinct exit codes so clients can easily key off those exit codes to influence logic on their end without parsing strings.

This could be separate ticket - just curious what you think.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good idea. Another option is to introduce an argument such as --format, similar to cargo's --message-format=json. It can provide detailed information about command success/failure, e.g. paths that successfully signed and paths that failed to sign. I think this would pair well with exit codes?

Here's a real example of how cargo outputs with --message-format=json (there are multiple outputs like this as the command runs):

{
   "reason":"compiler-artifact",
   "package_id":"registry+https://github.com/rust-lang/crates.io-index#[email protected]",
   "manifest_path":"/Users/me/.cargo/registry/src/index.crates.io-6f17d22bba15001f/mockall-0.12.1/Cargo.toml",
   "target":{
      "kind":[
         "lib"
      ],
      "crate_types":[
         "lib"
      ],
      "name":"mockall",
      "src_path":"/Users/me/.cargo/registry/src/index.crates.io-6f17d22bba15001f/mockall-0.12.1/src/lib.rs",
      "edition":"2021",
      "doc":true,
      "doctest":true,
      "test":true
   },
   "profile":{
      "opt_level":"0",
      "debuginfo":2,
      "debug_assertions":true,
      "overflow_checks":true,
      "test":false
   },
   "features":[
      
   ],
   "filenames":[
      "/Users/me/Documents/repos/c2patool/target/debug/deps/libmockall-7bdbc60eb0277649.rlib",
      "/Users/me/Documents/repos/c2patool/target/debug/deps/libmockall-7bdbc60eb0277649.rmeta"
   ],
   "executable":null,
   "fresh":true
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, it may help to restructure the program to remove its dependence on anyhow. If we provide detailed errors we can support detailed output formats ^ and assign error codes to errors via Termination.

@ok-nick ok-nick marked this pull request as ready for review June 17, 2024 18:49
@ok-nick ok-nick mentioned this pull request Jul 22, 2024
3 tasks
* Foundation for migrating to unstable API

* Extract resources using unstable API and introduce --unknown flag

* Use unstable API for viewing

* More unstable API during extraction

* Use c2pa-rs branch w/ wip features

* Set ingredient base paths

* Use `Manifest::signature_info` to output certs

* Migrate `view tree` command, rename `debug` flag to `detailed`, and update dependencies

* Fix `--unknown` flag when extracting resources

* Normalize URIs when extracting ingredients

* Rearrange

* Add `--no-embed` flag to signing`

* Small changes to tests and fix comments

* Use Reader in tests

* More strict uri normalization and fix typo
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants