Bring solfuzz-agave to the mono repo

[LucasSte]

Problem

The fuzzer we are developing together with Firedancer needs this glue code to work with Agave. The repository was previously outside the monorepo, hindering proper maintenance.

Summary of Changes

I brought the repository and made some adjustments to satisfy clippy and the dependencies path.

This is another workspace under Agave, as it is not a published crate and its compilation commands require specific settings.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 81.8%. Comparing base (57ab944) to head (34bf2cb).
Report is 6 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff            @@
##           master     #955     +/-   ##
=========================================
- Coverage    81.9%    81.8%   -0.1%     
=========================================
  Files         855      855             
  Lines      232154   232154             
=========================================
- Hits       190170   190129     -41     
- Misses      41984    42025     +41     

[pgarg66]

@LucasSte a few quick questions:

1. Were any changes made to the new files?
2. Can we find a better name for the top level folder? Just in case we have other fuzzers in the future.

[ripatel-fd]

We might want to rewrite this self-test in Rust in the future.
I just wrote it in C originally because it was the easiest way to get the test working.

[LucasSte]

Yes, we can do it in another PR. The .cargo/config.toml file may need adjustments as cargo pass down the flags to all compiler invocations, whereas it only uses the flag for the project when we use the --target x86_64-unknown-linux-gnu argument. I haven't taken the time to investigate further how to set different flags for tests.

[ripatel-fd]

In the future, we might want to move these Protobuf definitions to some shared place.
Maybe storage-proto?

Firedancer has functionality to create these test vectors from real ledgers (it allows us to debug complex program executions). Agave might want to add the same feature.

[LucasSte]

In the future, we might want to move these Protobuf definitions to some shared place. Maybe storage-proto?

I was going to suggest this as well. Replicating files in many repositories is not a good practice. Shall we have a separate repo and add it as a submodule here?

[ripatel-fd]

@LucasSte I suggest we use solana-foundation/specs. Let's also register the schema on https://buf.build/

[LucasSte]

@LucasSte a few quick questions:

1. Were any changes made to the new files?

I tried to bring the repository as it is now. I made small changes to make our clippy tests happy. There are separate commits to address the warnings I got. There was another PR merged in solfuzz-agave (firedancer-io/solfuzz-agave#3) with more changes, but I thought it was best to add them after that I merge this PR. The repository solfuzz-agave will be archived once we merge this PR.

2. Can we find a better name for the top level folder? Just in case we have other fuzzers in the future.

Yes. Do you have any suggestion? Solfuzz?

[LucasSte]

@LucasSte a few quick questions:

1. Were any changes made to the new files?

Forgot to mention:
I removed the original Makefile in favor of .cargo/config.toml and build.rs to align the style to our repository practices.

[t-nelson]

The repository was previously outside the monorepo, hindering proper maintenance.

can this be elaborated? we're trying to get stuff out of the monorepo, not put more in

[LucasSte]

The repository was previously outside the monorepo, hindering proper maintenance.

can this be elaborated? we're trying to get stuff out of the monorepo, not put more in

solfuzz is a fuzzer engine developed by the Firedancer team that can be used to fuzz the Agave validator, or run differential fuzzing between versions of Agave or between Firedancer and Agave. There has been an effort in concentrating the fuzzers we are developing in solfuzz. Sean and I are expanding it to cover the VM and the transaction processing APIs.

solfuzz is dynamically linked to a glue code we need to run the fuzzer either in Firedancer or in Agave. Firedancer has this glue in their own repository already, and we thought we could bring it closer to Agave, as it is going to be mostly used by Anza's engineers. The repository today is located in the Firedancer organization: https://github.com/firedancer-io/solfuzz-agave. There was an idea to bring it to the monorepo, but I think creating another repository under the Github Anza organization would also work.

@ripatel-fd Please, feel free to add more context if I forgot anything.

Now, @t-nelson we can create another repo under Anza. What do you think?

[yihau]

do you have any name suggestions if we decide to move it to another repository? btw, it looks like we can just fork from Firedancer and then make some custom modifications 🤔

[yihau]

looks like we haven't had a conclusion 🤔 I just forked the repo here: https://github.com/anza-xyz/solfuzz-agave in case we need it!

[LucasSte]

@t-nelson Few other points came to my mind during the weekend to justify moving the fuzzer to the monorepo. If any of the target APIs receive updates, the mono repo CI would point that the fuzzer needs the same updates. If the fuzzer is a separate repository, these updates would need to be manual.

In addition, selecting Agave's version to fuzz with the project on the monorepo is as simple as doing git checkout, while in an external repository we need to manually add the commit/version in the Cargo.toml file and make the necessary adaptations.

[LucasSte]

Closing the PR in favor of maintaining the fuzzer outside the monrepo, as a separate repository: https://github.com/anza-xyz/solfuzz-agave