Store interface

[fogti]

Regarding store abstractions, it would be nice to know / extrapolate what exactly is necessary for the interpreter side to deal with stores. Thus, I want to sort-of collect stuff here which I think may be important (it if gets too off-topic, I'll move stuff to https://github.com/RIIR-Nix/rfcs instead)
Also, the operations of the store can be split into three categories: Reading, Writing, Building. I think it might be a good idea to model this via multiple (maybe async) traits, especially because some stores don't support Writing (read-only) or Building (S3), e.g.

trait StoreRead { /* query store prefix, has hash path?, download hash path from store ... */ }
trait StoreWrite: StoreRead { /* upload hash path to store ... */ }
trait StoreBuild: StoreWrite { /* build derivation ... */ }

(derivation building is a bit weird, because the derivation function takes an attrset containing a pretty wild mix, and doesn't explicitly separate what ends up in the environment, what doesn't, might contain magic modifiers (__contentAddressed, __structuredAttrs,...), etc. (see also, somewhat generic abstraction of that))

Another question would be how to deal with store paths, store hashes and potentially content addressing, and the three different "types"/kinds of store paths/hashes:

• input addressed recursively based on input addressed only (this is usually the only type in classic non-CA Nix stores)
• input addressed with mixed (some CA-outputs as inputs) or no inputs)
• content addressed
  (these correspond roughly to the content-addressed derivation resolver in Nix; and leakage of placeholders in Nix when using content-addressed derivations sort-of proofs that this is a bit more difficult than expected)

Also, store paths have some sort of common format, which is useful when e.g. rewriting store paths in dumps or such (pretty much implemented here (although that store path format deviates a bit from the nix format, which is the reason why store_ref_scanner::StoreSpec exists))

Another thing to abstract are "store dumps" (e.g. deserialized NARs), e.g. something like https://github.com/YZITE/yzix/blob/cdd10d1249d6b6d8a79ed04d6acb01f92c932117/crates/yzix-core/src/dump.rs#L13-L17

pub enum Dump {
    Regular { executable: bool, contents: Vec<u8> },
    SymLink { target: Utf8PathBuf },
    Directory(std::collections::BTreeMap<crate::BaseName, Dump>),
}

[griff]

As for dealing with NARs the interface in Nix.rs is event driven so it can deal with NARs of any size without having to store the whole NAR in memory.
It is also complete in that has readers and writers for both NAR files and filesystem and is fully async.

[fogti]

oh, I forgot to mention another thing which is especially important for the builder backend: it needs to be able to bind-mount stuff. although that might be an detail which doesn't need to be represented in a general interface because usually stuff just gets copied beforehand.

I also added https://github.com/YZITE/yzix/blob/c19f56d36d14eec62e7d0daea873ade8b477fc38/crates/yzix-core/src/ser_trait.rs to yzix to be able to change the network serialization framework later without affecting hashes. Only disadvantage is that it isn't async... But adding streaming support should be realistic.