Code coverage 'permission denied' Nix devShell

[tmayoff]

I'm trying to run code coverage with a nix flake devShell and I cannot get it to work. It errors with either No such file or directory or permission denied as it tries searchig the root folder. The github actions log is linked below, any help would be appreciated

https://github.com/tmayoff/wren/actions/runs/11632039595/job/32394326170?pr=43

[eli-schwartz]

Permission denied sounds like it is somehow nix itself not letting gcovr access the files you want. I would probably ask the nix people how the sandbox works...

[tmayoff]

I will also ask on the nix forums but I don't think it's being sandoxed the devShell should run it with access to the to most files.

Is there information on how the coverage works, why is it looking for these files in a bunch of directories.

	(gcovr could not infer a working directory that resolved it.)
To ignore this error use option --gcov-ignore-errors=no_working_dir_found.
(ERROR) Error occurred while reading reports:
Traceback (most recent call last):
  File "/nix/store/xz0qx2f01d14jrjycl5vb1d72ah4mjxh-python3.12-gcovr-8.2/lib/python3.12/site-packages/gcovr/__main__.py", line 358, in main
    covdata: CovData = gcovr_formats.read_reports(options)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/xz0qx2f01d14jrjycl5vb1d72ah4mjxh-python3.12-gcovr-8.2/lib/python3.12/site-packages/gcovr/formats/__init__.py", line 85, in read_reports
    covdata = GcovHandler(options).read_report()
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/xz0qx2f01d14jrjycl5vb1d72ah4mjxh-python3.12-gcovr-8.2/lib/python3.12/site-packages/gcovr/formats/gcov/__init__.py", line 215, in read_report
    return read_report(self.options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/xz0qx2f01d14jrjycl5vb1d72ah4mjxh-python3.12-gcovr-8.2/lib/python3.12/site-packages/gcovr/formats/gcov/read.py", line 96, in read_report
    with Workers(
         ^^^^^^^^
  File "/nix/store/xz0qx2f01d14jrjycl5vb1d72ah4mjxh-python3.12-gcovr-8.2/lib/python3.12/site-packages/gcovr/formats/gcov/workers.py", line 177, in __exit__
    self.wait()
  File "/nix/store/xz0qx2f01d14jrjycl5vb1d72ah4mjxh-python3.12-gcovr-8.2/lib/python3.12/site-packages/gcovr/formats/gcov/workers.py", line 168, in wait
    raise self.exceptions[0][1]
  File "/nix/store/xz0qx2f01d14jrjycl5vb1d72ah4mjxh-python3.12-gcovr-8.2/lib/python3.12/site-packages/gcovr/formats/gcov/read.py", line 103, in read_report
    contexts = pool.wait()
               ^^^^^^^^^^^
  File "/nix/store/xz0qx2f01d14jrjycl5vb1d72ah4mjxh-python3.12-gcovr-8.2/lib/python3.12/site-packages/gcovr/formats/gcov/workers.py", line 168, in wait
    raise self.exceptions[0][1]
  File "/nix/store/xz0qx2f01d14jrjycl5vb1d72ah4mjxh-python3.12-gcovr-8.2/lib/python3.12/site-packages/gcovr/formats/gcov/workers.py", line 81, in worker
    work(*args, **kwargs)
  File "/nix/store/xz0qx2f01d14jrjycl5vb1d72ah4mjxh-python3.12-gcovr-8.2/lib/python3.12/site-packages/gcovr/formats/gcov/read.py", line 597, in process_datafile
    raise RuntimeError(errors_output)
RuntimeError: GCOV produced the following errors processing /home/tyler/src/wren/build/wren_math/tests/wren_math_geometry_test.p/geometry.cpp.gcda:
	In directory /home/tyler/src/wren:
../wren_math/tests/geometry.cpp: No such file or directory
/home/tyler/src/wren/build/include/boost/test/utils/basic_cstring/basic_cstring.hpp: No such file or directory
/home/tyler/src/wren/build/include/boost/test/unit_test_suite.hpp: No such file or directory
/home/tyler/src/wren/build/include/boost/test/unit_test.hpp: No such file or directory
/home/tyler/src/wren/build/include/boost/test/tree/test_unit.hpp: No such file or directory
/home/tyler/src/wren/build/include/boost/function/function_template.hpp: No such file or directory
/home/tyler/src/wren/build/include/boost/function/function_fwd.hpp: No such file or directory
../wren_math/include/wren/math/vector.hpp: No such file or directory
../wren_math/include/wren/math/matrix.hpp: No such file or directory
/home/tyler/src/wren/build/include/boost/test/tools/assertion.hpp: No such file or directory
/home/tyler/src/wren/build/include/boost/test/utils/lazy_ostream.hpp: No such file or directory
/home/tyler/src/wren/build/include/boost/test/tools/assertion_result.hpp: No such file or directory
/home/tyler/src/wren/build/include/boost/test/tools/detail/fwd.hpp: No such file or directory
/home/tyler/src/wren/build/include/boost/test/utils/wrap_stringstream.hpp: No such file or directory
/home/tyler/src/wren/build/include/boost/test/tree/fixture.hpp: No such file or directory
/home/tyler/src/wren/build/include/boost/function/function_base.hpp: No such file or directory
/home/tyler/src/wren/build/include/boost/smart_ptr/shared_ptr.hpp: No such file or directory
/home/tyler/src/wren/build/include/boost/smart_ptr/detail/shared_count.hpp: No such file or directory
/home/tyler/src/wren/build/include/boost/smart_ptr/detail/sp_counted_base_gcc_atomic.hpp: No such file or directory
/home/tyler/src/wren/build/include/boost/test/utils/basic_cstring/bcs_char_traits.hpp: No such file or directory
/home/tyler/src/wren/build/include/boost/test/utils/class_properties.hpp: No such file or directory
/home/tyler/src/wren/build/include/boost/test/tools/detail/print_helper.hpp: No such file or directory
../wren_math/include/wren/math/utils.hpp: No such file or directory
/home/tyler/src/wren/build/include/boost/smart_ptr/detail/operator_bool.hpp: No such file or directory
/home/tyler/src/wren/build/include/boost/test/utils/basic_cstring/io.hpp: No such file or directory

[eli-schwartz]

It's being done by:

gcovr \
    -r /home/runner/work/wren/wren \
    /home/runner/work/wren/wren/build \
    -e /home/runner/work/wren/wren/subprojects \
    -x \
    -o /home/runner/work/wren/wren/build/meson-logs/coverage.xml \
    --gcov-executable "llvm-cov gcov"

gcovr tries to find non-absolute referenced files anywhere in the search paths, if I recall correctly. Not all attempted paths are expected to work, only one of them has to.

[tmayoff]

Okay so I'm not sure about gcovr but I got lcov working. Is there a way to pass arguments into from meson/ninja lcov right now I have to run then manually like so:

lcov --directory . --capture --output-file build/coverage.info --gcov-tool ./tools/gcov.sh --ignore-errors source --ignore-errors inconsistent
lcov --remove build/coverage.info '*/tests/*' '/nix/*' --output-file build/coverage.info --ignore-errors source --ignore-errors inconsistent

The only thing now is that it's only showing the files (or library) that I have tests for as part of the code coverage making the % higher than it should be. https://app.codecov.io/github/tmayoff/wren/tree/main/?displayType=tree

[tmayoff]

Okay so this is the solution I've come to with lcov. I'm not sure if it's strictly a nix issue, as I first thought. I think I just rewrote what meson does to generate the coverage information but with the -ignore-errors source --ignore-errors inconsistent flags attached, if there's a way to pass those to meson it would be fine

coverage: test
    lcov --directory . --capture --initial --output-file build/initial.info --gcov-tool ./tools/gcov.sh --ignore-errors source --ignore-errors inconsistent
    lcov --directory . --capture --output-file build/coverage.info --gcov-tool ./tools/gcov.sh --ignore-errors source --ignore-errors inconsistent
    lcov --directory . -a build/coverage.info -a build/initial.info --output-file build/coverage.info --gcov-tool ./tools/gcov.sh --ignore-errors source --ignore-errors inconsistent
    lcov --remove build/coverage.info '*/tests/*' '/nix/*' --output-file build/coverage.info --ignore-errors source --ignore-errors inconsistent

[eli-schwartz]

If, in the root of your source tree, you have:

• a gcovr.cfg, it should automatically be picked up by gcovr
• a .lcovrc, meson's own coverage rule will ensure it is passed as an option when invoking lcov