Commit to FRI first layer

[andrewmilson]

Return opening positions and query positions per column size in FRI

Add FRI layer 0

[reviewable-StarkWare]

This change is 

[andrewmilson]

• Create proofs for Cairo #899
• Commit to FRI first layer #897 👈 (View in Graphite)
• Make CommitmentSchemeProver::prove_values take ownership #852 : 1 other dependent PR (#856 )
• Add arithmetic op counts to InfoEvaluator #832
• Add size estimate for StarkProof #831
• dev

This stack of pull requests is managed by Graphite. Learn more about stacking.

[github-actions]

⚠️ Performance Alert ⚠️

Possible performance regression was detected for benchmark.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 2.

Benchmark suite	Current: 2e1da82	Previous: f6214d1	Ratio
iffts/simd ifft/27	629093148 ns/iter (± 19830439)	312325137 ns/iter (± 5094310)	2.01
merkle throughput/simd merkle	31928540 ns/iter (± 632902)	14690867 ns/iter (± 434150)	2.17

This comment was automatically generated by workflow using github-action-benchmark.

CC: @spapinistarkware

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 94.88636% with 27 lines in your changes missing coverage. Please review.

Project coverage is 91.89%. Comparing base (3e5a81d) to head (1be06bd).

Files with missing lines	Patch %	Lines
crates/prover/src/core/prover/mod.rs	0.00%	14 Missing ⚠️
crates/prover/src/core/fri.rs	97.05%	9 Missing and 4 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##              dev     #897      +/-   ##
==========================================
- Coverage   91.95%   91.89%   -0.06%     
==========================================
  Files          93       93              
  Lines       13308    13251      -57     
  Branches    13308    13251      -57     
==========================================
- Hits        12237    12177      -60     
- Misses        954      961       +7     
+ Partials      117      113       -4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

---

🚨 Try these New Features:

• Flaky Tests Detection - Detect and resolve failed and flaky tests

[ilyalesokhin-starkware]

crates/prover/src/core/pcs/quotients.rs line 150 at r1 (raw file):

    for &query_position in query_positions {
        let domain_point = commitment_domain.at(bit_reverse_index(query_position, log_size));
        let row = row_index_iter.next().unwrap();

Suggestion:

    for (row, &query_position) in query_positions.iter().enumerate() {
        let domain_point = commitment_domain.at(bit_reverse_index(query_position, log_size));

[ilyalesokhin-starkware]

crates/prover/src/core/fri.rs line 658 at r1 (raw file):

    /// Stores the evaluations the verifier can't compute but needs in order to perform folding and
    /// verify the merkle decommitment.
    pub missing_evals: Vec<SecureField>,

wdyt?

Suggestion:

    /// values that the verifier needs but cannot deduce from previous computations, in the
    /// order they are needed.
    /// This complements the values that were queried. These must be supplied directly to
    /// the verifier.
    pub fri_witness: Vec<SecureField>,

[ilyalesokhin-starkware]

crates/prover/src/core/fri.rs line 131 at r1 (raw file):

pub struct FriProver<'a, B: FriOps + MerkleOps<MC::H>, MC: MerkleChannel> {
    config: FriConfig,
    first_layer: FriFirstLayerProver<'a, B, MC::H>,

not sure about the names since it includes all layers.

Code quote:

first_layer: FriFirstLayerProver<'a, B, MC::H>,

[ilyalesokhin-starkware]

crates/prover/src/core/pcs/quotients.rs line 157 at r1 (raw file):

            let eval = CircleEvaluation::new(
                domain,
                queried_values.take(domain.size()).copied().collect_vec(),

Here you have 2 eval points per query, right?

How does this work after the change?

Code quote:

 queried_values.take(domain.size()).copied().collect_vec(),

[andrewmilson]

Reviewable status: 0 of 11 files reviewed, 4 unresolved discussions (waiting on @ilyalesokhin-starkware and @shaharsamocha7)

---

crates/prover/src/core/fri.rs line 658 at r1 (raw file):

Previously, ilyalesokhin-starkware wrote…

wdyt?

LGTM

---

crates/prover/src/core/pcs/quotients.rs line 157 at r1 (raw file):

Previously, ilyalesokhin-starkware wrote…

Here you have 2 eval points per query, right?

How does this work after the change?

That was the case previously.
Now only 1 eval per query now with FRI layer 0

---

crates/prover/src/core/pcs/quotients.rs line 150 at r1 (raw file):

    for &query_position in query_positions {
        let domain_point = commitment_domain.at(bit_reverse_index(query_position, log_size));
        let row = row_index_iter.next().unwrap();

Done tyty

[andrewmilson]

Reviewable status: 0 of 11 files reviewed, 4 unresolved discussions (waiting on @ilyalesokhin-starkware and @shaharsamocha7)

---

crates/prover/src/core/fri.rs line 131 at r1 (raw file):

Previously, ilyalesokhin-starkware wrote…

not sure about the names since it includes all layers.

Open to suggestions.

I guess the way I think about it is that the first layer could includes all possible sizes. But all these polynomials are still commited and folded in a single layer (what I currently have called the first layer).

[ilyalesokhin-starkware]

crates/prover/src/core/fri.rs line 131 at r1 (raw file):

Previously, andrewmilson (Andrew Milson) wrote…

Open to suggestions.

I guess the way I think about it is that the first layer could includes all possible sizes. But all these polynomials are still commited and folded in a single layer (what I currently have called the first layer).

Ok, I see, so it's a commitment rather than an evaluation layer.

[ilyalesokhin-starkware]

crates/prover/src/core/fri.rs line 324 at r3 (raw file):

    // TODO(andrew): Consider making this an `Option` since there may be curcumstances where we
    // don't actually want to commit to the first layer evaluations. This is a simple change. FRI
    // just returns more query positions for the first layer.

I think this might be a per-log-size configuration rather than a global one.

Code quote:

    // TODO(andrew): Consider making this an `Option` since there may be curcumstances where we
    // don't actually want to commit to the first layer evaluations. This is a simple change. FRI
    // just returns more query positions for the first layer.

[ilyalesokhin-starkware]

Reviewed 3 of 11 files at r1, all commit messages.
Reviewable status: 2 of 11 files reviewed, 1 unresolved discussion (waiting on @shaharsamocha7)

[andrewmilson]

Reviewable status: 2 of 11 files reviewed, 1 unresolved discussion (waiting on @ilyalesokhin-starkware and @shaharsamocha7)

---

crates/prover/src/core/fri.rs line 324 at r3 (raw file):

Previously, ilyalesokhin-starkware wrote…

I think this might be a per-log-size configuration rather than a global one.

Updated the TODO. WDYT?

[ilyalesokhin-starkware]

Reviewed 6 of 11 files at r1, 2 of 3 files at r2, 1 of 1 files at r3.
Reviewable status: 8 of 11 files reviewed, all discussions resolved (waiting on @shaharsamocha7)