Best Practices for Hosting Policies for Verification Summary Attestations (VSAs)

[behnazh-w]

Background

Macaron is a checker and verifier tool that follows SLSA guidelines when possible for certain security properties. It discovers and supports in-toto provenances and attestations generated by slsa-github-generator, Witness, and GitHub's Attest Action (discovery is still WIP).

A feature added a few months ago is the generation of Verification Summary Attestation (VSA). We have been waiting for the VSA specification to stabilize before fully adhering to it. For example, we have not been populating the predicate.policy.uri and predicate.verifiedLevels fields. Instead we have added a custom field predicate.policy.content to inform the users what policy has been verified and let them conclude the level as appropriate.

Question

Now that the VSA specification appears to be reaching a good level of stability, I was wondering where policies should be hosted. One option is to host policies in Macaron's GitHub repository. But in my opinion, hosting the SLSA-related policies that different verifier tools can verify in a dedicated repository under slsa-framework organization would promote better discipline and encourage wider adoption. We can then reach a consensus on the appropriate value for predicate.verifiedLevels and use the policy URL in predicate.policy.uri field. Additionally, maintaining a repository of tools that support SLSA, similar to what CycloneDX is doing, provides better support for users. Is this a viable option?

[TomHennen]

Thanks for filing this!

Can you provide some examples of the policy you're thinking of? People can mean a lot of different things when they talk about 'policy'.

[colek42]

We support uploading Witness policies to Archivista. It returns a gitoid for the policy file which you can use as a URI/URL. Whatever policy format you are using may be supported out of the box, but we could help adding support if it does not.

[behnazh-w]

@TomHennen

Can you provide some examples of the policy you're thinking of? People can mean a lot of different things when they talk about 'policy'.

In Macaron, we use Datalog (Soufflé dialect) to define policies.

Here is an example:

Policy("has-verified-provenance", component_id, "Require a verified provenance file.") :-
    check_passed(component_id, "mcn_provenance_level_three_1"),
    check_passed(component_id, "mcn_provenance_derived_repo_1"),
    check_passed(component_id, "mcn_provenance_derived_commit_1").

apply_policy_to("has-verified-provenance", component_id) :-
    is_component(component_id, "pkg:pypi/[email protected]").

This policy requires the following three checks in Macaron to pass for an artifact with pkg:pypi/[email protected] PURL:

• mcn_provenance_level_three_1 checks if the artifact has a verifiable provenance at level 3
• mcn_provenance_derived_repo_1 checks if the source-code repository URL in the provenance matches the repository in the published metadata on PyPI
• mcn_provenance_derived_commit_1 checks if the commit tag for pypi/[email protected] on GitHub matches the commit in the provenance.

This policy would map to SLSA_BUILD_LEVEL_3.

See this tutorial for other examples.

[behnazh-w]

We support uploading Witness policies to Archivista. It returns a gitoid for the policy file which you can use as a URI/URL. Whatever policy format you are using may be supported out of the box, but we could help adding support if it does not.

@colek42 Thanks, that's very interesting. Would the Datalog policy shown above be supported?

[trishankatdatadog]

Hi @behnazh-w! Thanks for this very interesting issue. Would you like to join our community effort to define a standard in-toto policy language? 🙂

[trishankatdatadog]

Would the Datalog policy shown above be supported?

Our current community WIP reference implementation uses Google CEL to check attribute constraints, but we plan to make it extensible/pluggable so that end-users can use whatever engine they like (e.g., CEL, Rego, Datalog, Prolog).

[TomHennen]

Thanks for providing that example.

So, as I understand it, this is a general purpose policy for evaluating attestations and that it's not customized per user.

E.g. you're wondering where to store this bit

# The 'general purpose policy
Policy("has-verified-provenance", component_id, "Require a verified provenance file.") :-
    check_passed(component_id, "mcn_provenance_level_three_1"),
    check_passed(component_id, "mcn_provenance_derived_repo_1"),
    check_passed(component_id, "mcn_provenance_derived_commit_1").

And then when the user applies

# The 'project specific' policy
apply_policy_to("has-verified-provenance", component_id) :-
    is_component(component_id, "pkg:pypi/[email protected]").

You want to be able to point at the has-verified-provenance policy, but not the user specified bit that applies it to urllib3?

I have a couple of thoughts on what to do with the general purpose policy:

1. Hosting the general purpose policy in slsa-framework would seem fine?
2. Having someplace within slsa-framework that points to known implementations of SLSA and their policies would seem handy also? Then the general purpose policy would just be stored in your own location, but SLSA could refer people to it.
3. Would Archivista make sense here too? IDK
4. Some combination of the above.

The project specific policy though probably wouldn't go in slsa-framework that would get out of hand pretty quickly (each project will need to update their policies a lot and I don't think the SLSA maintainers want to be on the hook for that :)). That's where something like Archivista or a repo controlled by the project itself would make sense (which was demonstrated here IIUC https://github.com/slsa-framework/oss-na24-slsa-workshop).

[arewm]

Is this policy generic for multiple types of artifacts or is it specific to verifying the SLSA provenance for pypi artifacts? I am not that familiar with package distribution via pypi -- is this now-aging blog post still relevant or are there more recent developments?

For some build platforms, verifying SLSA 1.0 Build L3 is much simpler than this as it should just be associated with the builder.id in the provenance. As long as a policy verification trusts that builder id and the attestations that are made, then verification should be simpler.

That's not to say that policies are not interesting, especially considering that the use of policies has come up multiple times in community calls recently. As we continue investigating along this route, we will have to balance the desires to simplify SLSA (its comprehension, implementation, adoption, etc.) with the ability to accurately qualify the supply chain so that consumers can make decisions consistent with their appetite for risk.

Hosting the general purpose policy in slsa-framework would seem fine?

What would this look like if we extended to multiple different policy languages/engines? It is not likely that SLSA maintainers would have knowledge of the policies nor would they be able to determine whether the policy validly checks the requirements as they will not be familiar with all of the implementation details.

Having someplace within slsa-framework that points to known implementations of SLSA and their policies would seem handy also? Then the general purpose policy would just be stored in your own location, but SLSA could refer people to it.

I think that this makes sense. It is a shout-out to other tools that claim support for verifying SLSA provenance. Would it be something similar to the builder SLSA levels?

[behnazh-w]

@TomHennen

You want to be able to point at the has-verified-provenance policy, but not the user specified bit that applies it to urllib3?

Yes, that's right. We would point to the general purpose policy part and the user-specific part will be maintained by the project owner (e.g., urllib3 or packages that depend on it).

4. Some combination of the above.

My main concern is ensuring that each policy maps to a specific level, such as SLSA_BUILD_LEVEL_3, which would then be reported in the VSA. I believe a central repository where we can contribute and monitor such mappings of policies to their corresponding levels would be helpful. Without such mappings, consumers may find it difficult to use VSAs for delegation, unless they choose to trust the verifier.

Hosting the general purpose policy in slsa-framework would seem fine?

What would this look like if we extended to multiple different policy languages/engines? It is not likely that SLSA maintainers would have knowledge of the policies nor would they be able to determine whether the policy validly checks the requirements as they will not be familiar with all of the implementation details.

@arewm That's a fair point, but to make VSAs and the delegation of verification possible, we need to establish at least some mappings between the policies and claims (e.g., SLSA_BUILD_LEVEL_3). Otherwise, we would need users to trust tools like Macaron to provide these mappings.

[behnazh-w]

@arewm

Is this policy generic for multiple types of artifacts or is it specific to verifying the SLSA provenance for pypi artifacts?

Macaron's checks support various types of artifacts, and the policies themselves can be generic. For example, we support artifacts such as maven, npm, and others.

I am not that familiar with package distribution via pypi -- is this now-aging blog post still relevant or are there more recent developments?

Yes, this blog post is still relevant and the policy shown above verifies the provenance published using the mechanism described in the blog post as a GitHub release asset. What Macaron does in this case is that it takes a PURL, i.e., pkg:pypi/[email protected] and a policy as input and does the rest of the work automatically (discovering the provenance, identifying the expected repo URL and commit):

./run_macaron.sh analyze -purl pkg:pypi/[email protected] --skip-deps
./run_macaron.sh verify-policy --database output/macaron.db --file policy.dl

Having someplace within slsa-framework that points to known implementations of SLSA and their policies would seem handy also? Then the general purpose policy would just be stored in your own location, but SLSA could refer people to it.

I think that this makes sense. It is a shout-out to other tools that claim support for verifying SLSA provenance. Would it be something similar to the builder SLSA levels?

I was thinking more along the lines of a structured file, similar to CycloneDX's tools, which is rendered here and allows maintainers to add their tools.

[behnazh-w]

Would the Datalog policy shown above be supported?

Our current community WIP reference implementation uses Google CEL to check attribute constraints, but we plan to make it extensible/pluggable so that end-users can use whatever engine they like (e.g., CEL, Rego, Datalog, Prolog).

Thanks @trishankatdatadog. So, the idea would be to support Macaron's Datalog-based policies as constraints within a standard in-toto policy language, which could then be published in Archivista?

[trishankatdatadog]

So, the idea would be to support Macaron's Datalog-based policies as constraints within a standard in-toto policy language, which could then be published in Archivista?

Yes (ideally)!