Update flake.lock

[onekey-sec-bot]

Automated changes by the update-flake-lock GitHub Action.

Flake lock file updates:

• Updated input 'filter':
    'github:numtide/nix-filter/c843418ecfd0344ecb85844b082ff5675e02c443' (2023-12-04)
  → 'github:numtide/nix-filter/3449dc925982ad46246cfc36469baf66e1b64f17' (2024-01-15)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/63143ac2c9186be6d9da6035fa22620018c85932' (2024-01-02)
  → 'github:NixOS/nixpkgs/e5d1c87f5813afde2dda384ac807c57a105721cc' (2024-01-19)
• Updated input 'pyperscan':
    'github:vlaci/pyperscan/76f07b61c91309084a121bee292c138a8f96474a' (2023-12-15)
  → 'github:vlaci/pyperscan/6bb577e885bd03589af261291c9cb80b6e074b66' (2024-01-19)
• Updated input 'unblob-native':
    'github:onekey-sec/unblob-native/4dbd25ac361cabd6b55dc149c7a5648a3c703a17' (2024-01-08)
  → 'github:onekey-sec/unblob-native/79dd6a6a5cecfe4b60f6f3e8448a6242f17b65dc' (2024-01-18)
• Updated input 'unblob-native/advisory-db':
    'github:rustsec/advisory-db/a5fb72de318a74eb69a2c241c0e46705684a35d0' (2023-12-21)
  → 'github:rustsec/advisory-db/33acf3edda9c7b8a991b3a8007c1b2282932f262' (2024-01-17)
• Updated input 'unblob-native/crane':
    'github:ipetkov/crane/0efe36f9232e0961512572883ba9c995aa1f54b1' (2024-01-03)
  → 'github:ipetkov/crane/742170d82cd65c925dcddc5c3d6185699fbbad08' (2024-01-18)
• Updated input 'unblob-native/flake-utils':
    'github:numtide/flake-utils/4022d587cbbfd70fe950c1e2083a02621806a725' (2023-12-04)
  → 'github:numtide/flake-utils/1ef2e671c3b0c19053962c07dbda38332dcebf26' (2024-01-15)
• Updated input 'unblob-native/nix-filter':
    'github:numtide/nix-filter/c843418ecfd0344ecb85844b082ff5675e02c443' (2023-12-04)
  → 'github:numtide/nix-filter/3449dc925982ad46246cfc36469baf66e1b64f17' (2024-01-15)

Running GitHub Actions on this PR

GitHub Actions will not run workflows on pull requests which are opened by a GitHub Action.

To run GitHub Actions workflows on this PR, run:

git branch -D update_flake_lock_action
git fetch origin
git checkout update_flake_lock_action
git commit --amend --no-edit
git push origin update_flake_lock_action --force

New commits need reviewing

[qkaiser]

So Nix restored runtime dependency checking for python packages (see https://discourse.nixos.org/t/breaking-changes-announcement-for-unstable/17574/40?page=2).

We're hitting a wall with LIEF for two reasons:

• we can't upgrade LIEF because they do not provide wheels for python3.12 for versions after 0.12.3
• we cannot use pythonRelaxDepsHook because LIEF project structure is dumb (remember the postInstall and postBuild fixups), which makes python-relax-deps-hook fail:

lief> [739/739] Linking CXX shared library /build/source/api/python/build/lief.sondroid/pyVersion.cpp.oKKted.c.oe.cpp.o
lief> Copying /build/source/api/python/build/lief.so into build/lib.linux-x86_64-cpython-311/lief/_lief.cpython-311-x86_64-linux-gnu.so
lief> copying build/lib.linux-x86_64-cpython-311/lief/_lief.cpython-311-x86_64-linux-gnu.so -> lief
lief> running build_scripts
lief> creating build/scripts-3.11
lief> copying and adjusting examples/elf_reader.py -> build/scripts-3.11
lief> copying and adjusting examples/pe_reader.py -> build/scripts-3.11
lief> copying and adjusting examples/macho_reader.py -> build/scripts-3.11
lief> changing mode of build/scripts-3.11/elf_reader.py from 644 to 755
lief> changing mode of build/scripts-3.11/pe_reader.py from 644 to 755
lief> changing mode of build/scripts-3.11/macho_reader.py from 644 to 755
lief> /build/source/build
lief> /nix/store/7wwxxw7d04vfx1gjr0jyw0958zib890j-python-relax-deps-hook/nix-support/setup-hook: line 78: pushd: dist: No such file or directory
lief> /nix/store/v5irq7wvkr7kih0hhnch5nnv2dcq8c4f-stdenv-linux/setup: line 131: pop_var_context: head of shell_variables not a function context
error: builder for '/nix/store/1z7nxz8b8g59zbhq3pcn6g8fhr78vbyk-lief-0.13.2.drv' failed with exit code 1
error: 1 dependencies of derivation '/nix/store/0ny787y0sv65w49inlzj9pkv7z70bsgk-unblob-tests-24.1.4.drv' failed to build

[qkaiser]

Since I was working on upgrading some packages, I decided to take care of all of them. Pyright is now reporting more issues, but these are relevant and should be either handled or ignored depending on importance.

[qkaiser]

We were already talking about ditching LIEF as a dependency internally, this package issue might be a good excuse to kickstart this change.

[vlaci]

So Nix restored runtime dependency checking for python packages (see https://discourse.nixos.org/t/breaking-changes-announcement-for-unstable/17574/40?page=2).

We're hitting a wall with LIEF for two reasons:

* we can't upgrade LIEF because they do not provide wheels for python3.12 for versions after 0.12.3

* we cannot use `pythonRelaxDepsHook` because LIEF project structure is dumb (remember the `postInstall` and `postBuild` fixups), which makes `python-relax-deps-hook` fail:

I don't understand this. We need to relax in unblob:

diff --git a/nix/unblob/default.nix b/nix/unblob/default.nix
index 0be6c88..b2c0c0d 100644
--- a/nix/unblob/default.nix
+++ b/nix/unblob/default.nix
@@ -108,6 +108,7 @@ let
     pythonRelaxDeps = [
       "dissect.cstruct"
+      "lief"
       "structlog"
     ];
 
     makeWrapperArgs = [

This did the trick for me.

[vlaci]

Nitpick: lz4 and lark are updated in the same commit

We could also update dissect.cstruct to 3.x, as the nix derivation already uses 3.10 without issues and can get rid of one pythonRelaxDeps entry.

[vlaci]

Regarding pyright errors, I think the type hint in pyperscan's match callback is overzealous.
I could get rid of it by applying the following change there:

diff --git a/python/pyperscan/_pyperscan.pyi b/python/pyperscan/_pyperscan.pyi
index c5d1e64..142f554 100644
--- a/python/pyperscan/_pyperscan.pyi
+++ b/python/pyperscan/_pyperscan.pyi
@@ -176,12 +176,12 @@ class OnMatch(Protocol, Generic[_TContext_contra]):
     """Callback called on match."""
 
     def __call__(
-        self, context: _TContext_contra, tag: Any, start: int, end: Any
+        self, context: _TContext_contra, tag: Any, start: int, end: int, /
     ) -> Scan:
         """Called when a match happens.
 
         Note:
-            Call parameters are passed positonally.
+            Call parameters are passed positionally.
 
         Args:
             context: Object passed to [Database.build][pyperscan._pyperscan.Database.build]

I'd suggest ignoring those for now, and it will be fixed in an upcoming version.

[vlaci]

Looked into the pyright issues. Some of them were present before but wasn't reported, because they were disabled. In pyright 1.1.339 the default typeCheckingMode has changed from basic to the new standard setting¹. Setting it back to basic eliminates a new category of issues.
It makes sense to resolve these, but maybe not in a dependency update.

Footnotes

1. https://github.com/microsoft/pyright/blob/main/docs/configuration.md#diagnostic-rule-defaults ↩

[qkaiser]

Thanks for the pointers @vlaci ! Will look into it this week-end.

[qkaiser]

Nitpick: lz4 and lark are updated in the same commit

Split into two separate commits.

We could also update dissect.cstruct to 3.x, as the nix derivation already uses 3.10 without issues and can get rid of one pythonRelaxDeps entry.

Upgrading dissect.cstruct would force us to drop support for python 3.8. Maybe later.

Regarding pyright errors, I think the type hint in pyperscan's match callback is overzealous. I'd suggest ignoring those for now, and it will be fixed in an upcoming version.

Done in 42cdc5a

In pyright 1.1.339 the default typeCheckingMode has changed from basic to the new standard setting1. Setting it back to basic eliminates a new category of issues.

Set back to basic in 42cdc5a