[RFC]: Convert to direct-style with Eio

[patricoferris]

What's ported?

This draft PR ports parts of Irmin to https://github.com/ocaml-multicore/eio. This includes irmin, irmin-fs, irmin-pack, irmin-containers and irmin-chunk along with the tests. Most of the other libraries are not yet ported because of the effort to try and port the dependencies in particular graphql, ocaml-git and webmachine (although I think irmin-http might be on its way out).

I'm opening this draft PR to let people know of the existence of this work (in case they want to do some hacking ;)) and to also let people comment on it. It might be interesting to see if there are impacts on the benchmarks.

Some issues

One problem that has come up a few times, is Irmin's use of top-level values (even if lazy). In the Eio world this becomes a little tricky because they often need either access to some capability or a Eio.Switch.t. There is a port of irmin-watcher for example that requires the installation of a handler in order to be able to get a switch to the hooks. irmin-fs also requires something similar so that users would have to do this (see the fs unix tests):

let () =
  Eio_main.run @@ fun env ->
  Irmin_fs.run env#fs @@ fun () ->
  Irmin_watcher.run @@ fun () ->
  Irmin_test.Store.run "irmin-fs.unix" ~slow:false ~sleep:Eio_unix.sleep
       ~misc:[]
       [ (`Quick, Test_fs_unix.suite) ]

Which I'm definitely not a fan of! As pointed out in ocaml-multicore/eio#364 (comment) fiber-local storage could be used but comes with it's own problems. The Irmin_fs handler was more about getting the fs capability to the repo constructor that can only take a configuration because I wasn't sure how to do that otherwise. The only way to get a capability is if it is passed to you from Eio's standard environment.

Porting to Eio already is a massive breaking change, so perhaps some of this could be reworked to avoid all of this :)) Would love to know your thoughts?

[art-w]

Thanks a lot @patricoferris, this is fantastic work! We've rebased your branch and completed the missing libraries using lwt_eio here: https://github.com/art-w/irmin/tree/direct-style (... it got a bit messy, but all the tests are working now) Do you mind if I update your PR with the new patches? :)

[patricoferris]

Awesome! Sorry for the slow reply, please do whatever is easiest (pushing the changes here, new PR etc.)

[samoht]

Could we add temporary workarounds for the failing tests? I'd be very happy to merge that PR but I also would like to keep the CI green to detect future regressions.

[art-w]

Sure, that makes sense! I made some adjustments to turn the CI green, the only thing left is adding a Codecov upload token but I have neither access to our Codecov account or to the github secrets management. Can someone lend me the accesses or configure the secret such that I'll be able to update the coverage workflow? Thanks!