done with archive-opam #2

hannesm · 2024-12-03T12:14:08Z

this is a draft.

packages/FrontC/FrontC.3.4.3/opam

packages/arch-x86_32/arch-x86_32.1/opam

kit-ty-kate

lgtm

mseri · 2024-12-05T18:35:06Z

Thanks a lot. I think it’s almost there. Can we prevent removing the comments, as in avoiding things like

-available: false # uses a git url, no checksum. newer versions exist
+available: false

I hope that will not be a pain 🙏

The diff file is super useful, thanks again

hannesm · 2024-12-05T18:48:13Z

@mseri thanks for the suggestion, unfortunately, the opam-file-format package with the opam file parser does not preserve comments. While I see that it'd be great, it'd need adjustments to opam-file-format (and eventually opam itself). not sure whether it is worth it. I'll take a brief look into OpamFile.OPAM.to_string_with_preserved_format whether I can find ways to skip over comments / retain them.

hannesm · 2024-12-05T21:04:48Z

I force-pushed another commit, now with a hacked up archive-opam preserving any comment on the last line. As usual, the diff:

2024-12-05-d739816a52523fd016f6d208709ce47256f11896.txt

kit-ty-kate · 2024-12-12T10:55:18Z

As mentioned in ocaml/opam#6335, you shouldn't use opam admin check without -i. With -i only a 50-ish packages will be listed if i recall correctly. I believe most of those packages are actually uninstallable but there might be false-positives so it's always to be taken with a grain of salt and checked manually afterwards

mseri · 2024-12-13T10:46:56Z

Thanks for doing this. In my opinion it would make sense to move these packages alongside the available: false ones.
Maybe one exception, I will send a fix to dune-rpc-lwt 3.0.3 and see if that works once the bound is fixed.

How hard/computationally-expensive would it be to have a tool (or extend opam admin check) to attach the reason of the uninstallability to the opam file? Say as a comment at the end of the file or as a separate log file?

avsm · 2024-12-13T11:14:57Z

Wow, this list is really useful with those explanations attached, @hannesm! For example, I could see that we might be able to fix vchan-unix by fixing vchan. OTOH, I would also be fine with moving all these enmass to the archive repository, and letting interested maintainers propose them back to the mainline opam-repo once they're fixed.

We also need to restore the "uninstallability" check to the opam-repo-ci don't we? That got dropped for some reason a few years ago, but it's necessary to make sure that any new constraint changes don't make existing packages uninstallable.

hannesm · 2024-12-13T11:23:36Z

Ok, another try. Please disregard the above output. This one is better:

async_core.108.00.01 not satisfied: core = 108.00.01
bap-ida.2.4.0 not satisfied: bap-ida-python = 2.4.0
bap-ida.2.5.0 not satisfied: bap-ida-python = 2.5.0
cohttp-async.2.5.2 not satisfied: cohttp = version
cohttp-async.2.5.3 not satisfied: cohttp = version
cohttp-lwt.2.5.2 not satisfied: cohttp = version
cohttp-lwt.2.5.3 not satisfied: cohttp = version
cohttp-top.2.5.2 not satisfied: cohttp = version
cohttp-top.2.5.3 not satisfied: cohttp = version
dns-cli.4.6.0 not satisfied: mirage-crypto-rng >= 0.7.0 & < 0.8.0
dune-rpc-lwt.3.0.3 not satisfied: result >= 1.5.0
eio_main.0.1 not satisfied: eio_luv = version
eio_main.0.2 not satisfied: eio_luv = version
eio_main.0.3 not satisfied: eio_luv = version
eio_main.0.4 not satisfied: eio_luv = version
eio_main.0.5 not satisfied: eio_luv = version
eio_main.0.6 not satisfied: eio_luv = version
eio_main.0.7 not satisfied: eio_luv = version
eio_main.0.8.1 not satisfied: eio_luv = version
iocaml-kernel.0.4.0 not satisfied: zmq = 3.2-2
irmin-bench.2.7.1 not satisfied: progress = 1.1.0
js_of_ocaml.3.7.0 not satisfied: js_of_ocaml-compiler = version
mirage-block-combinators.2.0.0 not satisfied: mirage-block = version
mirage-clock-freestanding.3.0.0 not satisfied: mirage-clock = version
mirage-clock-unix.3.0.0 not satisfied: mirage-clock = version
mirage-conduit.2.1.0 not satisfied: vchan < 2.3.0
mirage-console-unix.3.0.0 not satisfied: mirage-console = version
mirage-console-unix.3.0.1 not satisfied: mirage-console = version
mirage-console-xen-proto.3.0.0 not satisfied: mirage-console = version
mirage-console-xen-proto.3.0.1 not satisfied: mirage-console = version
mirage-crypto-pk.0.7.0 not satisfied: mirage-crypto-rng = version
mirage-crypto-rng.0.8.1 not satisfied: mirage-crypto = version
mirage-crypto-rng.0.8.2 not satisfied: mirage-crypto = version
mirage-crypto-rng.0.8.3 not satisfied: mirage-crypto = version
mirage-crypto-rng.0.8.4 not satisfied: mirage-crypto = version
mirage-crypto-rng.1.0.0 not satisfied: mirage-crypto = version
mirage-entropy-xen.0.2.0 not satisfied: mirage-xen >= 2.2.0 & < 3.0.0
mirage-entropy-xen.0.3.0 not satisfied: mirage-xen >= 2.2.0 & < 3.0.0
mirage-flow-combinators.2.0.0 not satisfied: mirage-flow = version
mirage-flow-unix.2.0.0 not satisfied: mirage-flow = version
mirage-qubes-ipv4.0.9.4 not satisfied: mirage-qubes = version
mirage-qubes.0.1 not satisfied: vchan >= 2.0.0 & < 3.0.0
mirage-qubes.0.2 not satisfied: vchan >= 2.0.0 & < 3.0.0
mirage-qubes.0.3 not satisfied: vchan >= 2.0.0 & < 3.0.0
mirage-qubes.0.4 not satisfied: vchan >= 2.3.0 & < 3.0.0
mirage-tcpip-xen.0.9.5 not satisfied: mirage-xen = 0.9.9
mirage-time-unix.2.0.0 not satisfied: mirage-time = version
ocaml-top.1.0.0 not satisfied: ocp-build >= 1.99.6-beta & < 1.99.8-beta
ocaml-top.1.0.1 not satisfied: ocp-build >= 1.99.6-beta & < 1.99.8-beta
ocaml-top.1.1.0 not satisfied: ocp-build >= 1.99.6-beta & < 1.99.8-beta
octez.15.1 not satisfied: octez-accuser-PtKathma = version
oidc.0.1.0 not satisfied: jose >= 0.5.1 & < 0.8.0
oidc.0.1.1 not satisfied: jose >= 0.5.1 & < 0.8.0
opam-core.2.0.0~rc not satisfied: ocaml >= 4.02.3 & < 4.0.7
ospec.0.3.0 not satisfied: ocaml = broken
owl.0.2.0 not satisfied: eigen <= 0.0.3
owl.0.2.1 not satisfied: eigen <= 0.0.3
owl.0.2.2 not satisfied: eigen < 0.0.4
owl.0.2.3 not satisfied: eigen < 0.0.4
owl.0.2.4 not satisfied: eigen < 0.0.4
ox.1.0.0 not satisfied: openflow = 0.1.0
ox.1.0.1 not satisfied: openflow = 0.1.0
oxylc.alpha1.1.2 not satisfied: llvm = 7.0
qocamlbrowser.0.2.8 not satisfied: lablqml < 0.5
release.1.0.2 not satisfied: ocaml = broken
scaml.1.2.0 not satisfied: tezos-error-monad = 7.0
scaml.1.2.0~pre7 not satisfied: tezos-error-monad = 7.0
sociaml-tumblr-api.0.1.0 not satisfied: sociaml-oauth-client < 0.5.0
stog-rdf.0.15.0 not satisfied: rdf = 0.9.0
stog-rdf.0.16.0 not satisfied: rdf = 0.9.0
stog-writing.0.13.0 not satisfied: stog >= 0.13.0 & <= 0.14
tezos-client-base.10.2 not satisfied: tezos-sapling = version
tezos-client-base.12.3 not satisfied: tezos-sapling = version
tezos-error-monad.9.7 not satisfied: tezos-stdlib = version
tezos-protocol-environment.10.2 not satisfied: tezos-sapling = version
tezos-protocol-environment.12.3 not satisfied: tezos-sapling = version
tezt-tezos.17.1 not satisfied: tezos-crypto-dal = version
tezt-tezos.17.2 not satisfied: tezos-crypto-dal = version
torch.v0.17.1 not satisfied: libtorch >= 2.3.0 & < 2.4.0
utop.1.6 not satisfied: ocaml = 4.01
vchan-unix.3.0.0 not satisfied: vchan = 3.0.0
vchan-unix.4.0.0 not satisfied: vchan = 4.0.0
vchan-unix.4.0.1 not satisfied: vchan >= 4.0.0 & < 4.0.2
vchan-xen.3.0.0 not satisfied: vchan = 3.0.0
vchan-xen.4.0.0 not satisfied: vchan = 4.0.0
vchan-xen.4.0.1 not satisfied: vchan >= 4.0.0 & < 4.0.2
weevil.0.1.1 not satisfied: tezos-base = 15.0
xen-block-driver.0.2.0 not satisfied: mirage-xen = 0.9.2
xen-block-driver.0.2.2 not satisfied: mirage-xen >= 0.9.4 & < 0.9.8
xen-block-driver.0.2.3 not satisfied: mirage-xen = 0.9.7
xen-block-driver.0.2.4 not satisfied: mirage-xen = 0.9.7
xen-block-driver.0.2.5 not satisfied: mirage-xen = 0.9.7

What we can for sure fix: dune-rpc-lwt.3.0.3, utop.1.6, opam-core.2.0.0~rc

hannesm · 2024-12-13T11:34:24Z

and to add to the list above:

malfunction.0.2 not satisfied: base-flambda
malfunction.0.2.1 not satisfied: base-flambda

the only thing is ocaml-option-spacetime -- a dead package that has been introduced for the future --let's leave it as is ;)

hannesm · 2024-12-13T11:50:30Z

Wow, this list is really useful with those explanations attached, @hannesm! For example, I could see that we might be able to fix vchan-unix by fixing vchan.

The story about vchan-* is:

commit f1fb1682b4c4b635a95e71e991b1d19fb369b0e8 (mine/msa02)
Author: Hannes Mehnert <[email protected]>
Date:   Wed Oct 2 13:55:59 2019 +0200

    mark insecure packages (MSA02 https://mirage.io/blog/MSA02) unavailable:
    - mirage-xen < 3.4.0
    - mirage-block-xen < 1.6.1
    - mirage-net-xen < 1.10.2
    - mirage-console < 2.4.2
    - ocaml-vchan < 4.0.2

So, it turns out the "leftovers", vchan-unix < 4.0.2 and vchan-xen < 4.0.2, are still in the opam-repository, but not installable.

OTOH, I would also be fine with moving all these enmass to the archive repository, and letting interested maintainers propose them back to the mainline opam-repo once they're fixed.

YES

We also need to restore the "uninstallability" check to the opam-repo-ci don't we? That got dropped for some reason a few years ago, but it's necessary to make sure that any new constraint changes don't make existing packages uninstallable.

Indeed, we talked about this yesterday in the opam-repository meeting. There's an issue open ocurrent/opam-repo-ci#118

hannesm · 2024-12-13T12:30:12Z

And now, the second set (481 that depended on the uninstallable ones):

"async_extra.108.00.01" unsatisfied: "async_core" { = "108.00.01" }
"async_unix.108.00.01" unsatisfied: "async_core" { = "108.00.01" }
"async.108.00.01" unsatisfied: "async_core" { = "108.00.01" }
"bap-fsi-benchmark.2.4.0" unsatisfied: "bap-ida" { = "2.4.0" }
"bap-fsi-benchmark.2.5.0" unsatisfied: "bap-ida" { = "2.5.0" }
"bap.0.9.3" unsatisfied: "ocaml" { = "4.01" }
"bap.0.9.4" unsatisfied: "ocaml" { = "4.01" }
"bap.0.9.6" unsatisfied: "ocaml" { = "4.01" }
"cohttp-lwt-jsoo.2.5.2" unsatisfied: "cohttp" { = version }
"cohttp-lwt-jsoo.2.5.3" unsatisfied: "cohttp" { = version }
"cohttp-lwt-unix.2.5.2" unsatisfied: "cohttp-lwt" { = version }
"cohttp-lwt-unix.2.5.3" unsatisfied: "cohttp-lwt" { = version }
"eris.1.0.0" unsatisfied: "monocypher" { }
"git-mirage.3.7.1" unsatisfied: "git" { = version }
"git-mirage.3.8.1" unsatisfied: "git" { = version }
"git-paf.3.7.1" unsatisfied: "git" { = version }
"git-paf.3.8.1" unsatisfied: "git" { = version }
"git-unix.3.7.1" unsatisfied: "git" { = version }
"git-unix.3.8.1" unsatisfied: "git" { = version }
"iocaml.0.4.2" unsatisfied: "iocaml-kernel" { = "0.4.0" }
"js_of_ocaml-lwt.3.7.0" unsatisfied: "js_of_ocaml" { = version }
"js_of_ocaml-ppx_deriving_json.3.7.0" unsatisfied: "js_of_ocaml" { = version }
"js_of_ocaml-ppx.3.7.0" unsatisfied: "js_of_ocaml" { = version }
"js_of_ocaml-toplevel.3.7.0" unsatisfied: "js_of_ocaml-compiler" { = version }
"js_of_ocaml-tyxml.3.7.0" unsatisfied: "js_of_ocaml" { = version }
"mirage-console-xen-backend.3.0.0" unsatisfied: "mirage-console" { = version }
"mirage-console-xen-backend.3.0.1" unsatisfied: "mirage-console" { = version }
"mirage-console-xen.3.0.0" unsatisfied: "mirage-console" { = version }
"mirage-console-xen.3.0.1" unsatisfied: "mirage-console" { = version }
"mirage-crypto-ec.1.0.0" unsatisfied: "mirage-crypto-rng" { = version }
"mirage-crypto-pk.0.8.1" unsatisfied: "mirage-crypto" { = version }
"mirage-crypto-pk.0.8.2" unsatisfied: "mirage-crypto" { = version }
"mirage-crypto-pk.0.8.3" unsatisfied: "mirage-crypto" { = version }
"mirage-crypto-pk.0.8.4" unsatisfied: "mirage-crypto" { = version }
"mirage-crypto-pk.1.0.0" unsatisfied: "mirage-crypto" { = version }
"mirage-crypto-rng-async.1.0.0" unsatisfied: "mirage-crypto-rng" { = version }
"mirage-crypto-rng-eio.1.0.0" unsatisfied: "mirage-crypto-rng" { = version }
"mirage-crypto-rng-lwt.1.0.0" unsatisfied: "mirage-crypto-rng" { = version }
"mirage-crypto-rng-miou-unix.1.0.0" unsatisfied: "mirage-crypto-rng" { = version }
"mirage-crypto-rng-mirage.0.8.1" unsatisfied: "mirage-crypto-rng" { = version }
"mirage-crypto-rng-mirage.0.8.2" unsatisfied: "mirage-crypto-rng" { = version }
"mirage-crypto-rng-mirage.0.8.3" unsatisfied: "mirage-crypto-rng" { = version }
"mirage-crypto-rng-mirage.0.8.4" unsatisfied: "mirage-crypto-rng" { = version }
"mirage-crypto-rng-mirage.1.0.0" unsatisfied: "mirage-crypto-rng" { = version }
"mirage-http-xen.1.0.0" unsatisfied: "mirage-tcpip-xen" { >= "0.9.5" }
"scaml.1.5.0" unsatisfied: "tezos-error-monad" { = "14.0" }
"tezos-008-PtEdo2Zk-test-helpers.10.2" unsatisfied: "tezos-protocol-environment" { = version }
"tezos-009-PsFLoren-test-helpers.10.2" unsatisfied: "tezos-protocol-environment" { = version }
"tezos-010-PtGRANAD-test-helpers.10.2" unsatisfied: "tezos-protocol-environment" { = version }
"tezos-client-base.9.7" unsatisfied: "tezos-sapling" { = version }
"tezos-client.10.2" unsatisfied: "tezos-node" { with-test & = version }
"tezos-client.12.3" unsatisfied: "tezos-node" { with-test & = version }
"tezos-client.9.7" unsatisfied: "tezos-node" { with-test & = version }
"tezos-event-logging.9.7" unsatisfied: "tezos-error-monad" { = version }
"tezos-legacy-store.10.2" unsatisfied: "tezos-lmdb" { = "7.4" }
"tezos-micheline.9.7" unsatisfied: "tezos-error-monad" { = version }
"tezos-mockup-proxy.10.2" unsatisfied: "tezos-client-base" { = version }
"tezos-mockup-proxy.12.3" unsatisfied: "tezos-client-base" { = version }
"tezos-mockup-registration.10.2" unsatisfied: "tezos-client-base" { = version }
"tezos-mockup-registration.12.3" unsatisfied: "tezos-client-base" { = version }
"tezos-protocol-compiler.10.2" unsatisfied: "tezos-protocol-environment" { = version }
"tezos-protocol-compiler.12.3" unsatisfied: "tezos-protocol-environment" { = version }
"tezos-protocol-environment.9.7" unsatisfied: "tezos-sapling" { = version }
"tezos-rpc.9.7" unsatisfied: "tezos-error-monad" { = version }
"tezos-shell-context.10.2" unsatisfied: "tezos-protocol-environment" { = version }
"tezos-shell-context.12.3" unsatisfied: "tezos-protocol-environment" { = version }
"tezos-signer-services.10.2" unsatisfied: "tezos-client-base" { = version }
"tezos-signer-services.12.3" unsatisfied: "tezos-client-base" { = version }
"tezos-storage.9.7" unsatisfied: "tezos-lmdb" { = "7.4" }
"tls-mirage.0.12.1" unsatisfied: "tls" { = version }
"xen-disk.1.0.2" unsatisfied: "xen-block-driver" { >= "0.2.0" }
"xen-disk.1.0.3" unsatisfied: "xen-block-driver" { >= "0.2.0" }
"xen-disk.1.2.1" unsatisfied: "mirage-block-xen" { >= "1.1.0" & < "1.2.0" }

I'm pretty sure there isn't yet a fixpoint reached, but will need to have a break from opam-repo-archive.

hannesm · 2024-12-13T16:05:09Z

thanks for discussions, closing this in favour of the real one [tm] for phase 1. if you have any open comments/suggestions, please get in touch.

hannesm force-pushed the initial-removal branch from 174fceb to 2d0aa92 Compare December 3, 2024 12:16

hannesm marked this pull request as draft December 3, 2024 12:23

This comment was marked as outdated.

Sign in to view

hannesm changed the title ~~done with archive-opam 2ac5b4411dc6433623d35bbb1ad092b393d3174e~~ done with archive-opam e285d7b4abb8c9fc12c1b55e397ff21aa22dd8b0 Dec 3, 2024

kit-ty-kate reviewed Dec 3, 2024

View reviewed changes

packages/FrontC/FrontC.3.4.3/opam Show resolved Hide resolved

kit-ty-kate reviewed Dec 3, 2024

View reviewed changes

packages/arch-x86_32/arch-x86_32.1/opam Outdated Show resolved Hide resolved

hannesm force-pushed the initial-removal branch from 2d0aa92 to c11482d Compare December 4, 2024 12:09

hannesm changed the title ~~done with archive-opam e285d7b4abb8c9fc12c1b55e397ff21aa22dd8b0~~ done with archive-opam c81c1bb6d92044f80834d4d3748faec7a6b25015 Dec 4, 2024

This comment was marked as outdated.

Sign in to view

kit-ty-kate approved these changes Dec 4, 2024

View reviewed changes

hannesm force-pushed the initial-removal branch from c11482d to 98e2d4f Compare December 5, 2024 16:55

This comment was marked as outdated.

Sign in to view

hannesm changed the title ~~done with archive-opam c81c1bb6d92044f80834d4d3748faec7a6b25015~~ done with archive-opam Dec 5, 2024

with archive-opam d739816a52523fd016f6d208709ce47256f11896

7daa6d0

hannesm force-pushed the initial-removal branch from 98e2d4f to 7daa6d0 Compare December 5, 2024 21:02

This comment was marked as off-topic.

Sign in to view

This comment was marked as outdated.

Sign in to view

mseri mentioned this pull request Dec 13, 2024

dune-rpc-lwt 3.0.3 has an incorrect bound on result ocaml/opam-repository#27094

Merged

This comment was marked as outdated.

Sign in to view

hannesm mentioned this pull request Dec 13, 2024

fix ocaml constraints that were mistyped ocaml/opam-repository#27096

Merged

hannesm mentioned this pull request Dec 13, 2024

fix ocaml constraint in bap 0.9.{3,4,6} ocaml/opam-repository#27097

Merged

hannesm closed this Dec 13, 2024

hannesm deleted the initial-removal branch December 13, 2024 16:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

done with archive-opam #2

done with archive-opam #2

hannesm commented Dec 3, 2024

This comment was marked as outdated.

This comment was marked as outdated.

kit-ty-kate left a comment

This comment was marked as outdated.

mseri commented Dec 5, 2024 •

edited

Loading

hannesm commented Dec 5, 2024

hannesm commented Dec 5, 2024

This comment was marked as off-topic.

kit-ty-kate commented Dec 12, 2024

This comment was marked as outdated.

mseri commented Dec 13, 2024

This comment was marked as outdated.

This comment was marked as outdated.

avsm commented Dec 13, 2024

hannesm commented Dec 13, 2024

hannesm commented Dec 13, 2024

hannesm commented Dec 13, 2024

hannesm commented Dec 13, 2024

hannesm commented Dec 13, 2024

done with archive-opam #2

done with archive-opam #2

Conversation

hannesm commented Dec 3, 2024

This comment was marked as outdated.

This comment was marked as outdated.

kit-ty-kate left a comment

Choose a reason for hiding this comment

This comment was marked as outdated.

mseri commented Dec 5, 2024 • edited Loading

hannesm commented Dec 5, 2024

hannesm commented Dec 5, 2024

This comment was marked as off-topic.

kit-ty-kate commented Dec 12, 2024

This comment was marked as outdated.

mseri commented Dec 13, 2024

This comment was marked as outdated.

This comment was marked as outdated.

avsm commented Dec 13, 2024

hannesm commented Dec 13, 2024

hannesm commented Dec 13, 2024

hannesm commented Dec 13, 2024

hannesm commented Dec 13, 2024

hannesm commented Dec 13, 2024

mseri commented Dec 5, 2024 •

edited

Loading