[BUG] Heavy flow of errors with NixD #2390

Sk7Str1p3 · 2024-10-08T17:05:12Z

Field	Description
Plugin	`lspconfig`
Nixpkgs	`unstable`
Home Manager	`unstable`

I have read the FAQ and my bug is not listed there.

Description

error i get every time editing configuration.nix

Logs

[START][2024-10-08 20:06:15] LSP logging initiated
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I[20:06:15.636] 16509: <-- initialize(1)\n"
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I[20:06:15.636] 16509: --> reply:initialize(1)\n"
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I[20:06:15.637] 16509: --> call window/workDoneProgress/create(1)\nI[20:06:15.637] 16509: --> notify $/progress\nI[20:06:15.637] 16509: --> call attrset/evalExpr(1)\nI[20:06:15.637] 16509: --> call window/workDoneProgress/create(2)\nI[20:06:15.637] 16509: --> notify $/progress\nI[20:06:15.637] 16509: --> call attrset/evalExpr(1)\nI[20:06:15.637] 16509: --> call workspace/configuration(3)\nI[20:06:15.638] 16509: <-- initialized\nI[20:06:15.638] 16509: <-- textDocument/didOpen\n"
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I[20:06:15.639] 16509: --> notify textDocument/publishDiagnostics\n"
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I[20:06:15.639] 16509: <-- textDocument/inlayHint(2)\n"
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I[20:06:15.639] 16509: <-- textDocument/inlayHint(3)\nI[20:06:15.639] 16509: <-- reply(1)\nI[20:06:15.639] 16509: --> call attrset/attrpathInfo(2)\nI[20:06:15.639] 16509: --> call attrset/attrpathInfo(3)\n"
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I[20:06:15.639] 16509: <-- reply(2)\n"
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I[20:06:15.640] 16509: <-- reply(3)\nE[20:06:15.640] 16509: workspace/configuration: parse error expected object\n"
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I[20:06:15.640] 16509: <-- textDocument/semanticTokens/full(4)\n"
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I[20:06:15.643] 16509: --> reply:textDocument/semanticTokens/full(4)\n"
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I[20:06:15.793] "
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"16509: <-- reply(1)\nI[20:06:15.793] 16509: --> notify $/progress\n"
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I[20:06:15.794] 16509: <-- reply(2)\n"
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I[20:06:15.794] 16509: <-- reply(3)\n"
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I["
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"20:06:15.794] 16509: --> reply:textDocument/inlayHint(2)\n"
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I[20:06:15.794] 16509: --> reply:textDocument/inlayHint(3)\n"
[ERROR][2024-10-08 20:06:16] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I[20:06:16.157] 16509: <-- reply(1)\nI[20:06:16.157] 16509: --> notify $/progress\n"

Minimal, Reproducible Example (MRE)

programs.nixvim = {
  enable = true;
  lsp = {
	    enable = true;
	    inlayHints = true;
	    servers = {
	      lua-ls = {
	        enable = true;
	      };
	      pylsp = {
	        enable = true;
	      };
	      nixd = {
	        enable = true;
	      };
	      clangd = {
	        enable = true;
	      };
	    };
	  };
}

also nixd installed with environment.systemPackages. except flow of errors, lsp works fine

The text was updated successfully, but these errors were encountered:

GaetanLepage · 2024-10-08T17:11:33Z

@inclyc do you think this issue is on our side ?
FWIW, neovim has been updated to 0.10.2, but I would be surprised if it would break something like this.

inclyc · 2024-10-09T03:23:28Z

According to nixd's log I think it works as expected, can you take a look? @GaetanLepage

Sk7Str1p3 · 2024-10-09T06:45:36Z

Yes, as I said lsp works fine

khaneliman · 2024-10-09T13:21:20Z

Sounds similar to report in #2290 . It doesn't seem to affect everyone so I'm not sure of the root cause, yet.

Sk7Str1p3 · 2024-10-10T04:19:46Z

UPD someone figured out that issues appears only if nerdfont's glyths are used in file

MattSturgeon · 2024-10-10T23:56:25Z

issues appears only if nerdfont's glyths are used in file

Interesting, I haven't had the issue with other unicode characters... I do sometimes include emoji in my nix comments.

It is worth noting that nix strings themselves don't have UTF-8 support, so if nix evaluates a unicode string you can run into issues. IDK if nixd actually evaluates any strings using nix, but maybe they have similar bugs on their end?

Lassulus · 2024-10-11T16:40:53Z

for me this happens (recently? not sure I had this before) when editing files which have unicode in them. In a nix config I have this line:
https://github.com/Lassulus/superconfig/blob/master/tools/nvim.nix#L26

listchars = "tab:⇥ ,extends:❯,precedes:❮,nbsp:␣,trail:·,eol:↲";

and now nixvim with nixd throws errors when opening the file which make editing pretty hard.

eljamm · 2024-10-11T19:55:30Z

I can reproduce this issue with neovim 0.10.2, but everything works fine with 0.10.0 and 0.10.1.

Sk7Str1p3 · 2024-10-11T20:16:05Z

So this is neovim problem? Gotta open issue in their repo.

eljamm · 2024-10-11T20:21:44Z

Yes, I've bisected the issue to neovim/neovim@b55435f

bisect log

git bisect start
# status: waiting for both good and bad commits
# good: [7e194f0d0c33a0a1b7ccfaf2baafdacf7f22fbb5] NVIM 0.10.1
git bisect good 7e194f0d0c33a0a1b7ccfaf2baafdacf7f22fbb5
# status: waiting for bad commit, 1 good commit known
# bad: [8b98642002d0506d20628683958cb5c97a0dad80] NVIM 0.10.2
git bisect bad 8b98642002d0506d20628683958cb5c97a0dad80
# good: [1fe8dc4d2c4e464684bfa05350e69c47c31431e7] Merge pull request #30211 from neovim/backport-30209-to-release-0.10
git bisect good 1fe8dc4d2c4e464684bfa05350e69c47c31431e7
# bad: [ba47b440fdf975a26c267337bc3a56c75349488e] fix(treesitter): lint top-level anonymous nodes
git bisect bad ba47b440fdf975a26c267337bc3a56c75349488e
# bad: [6f34f07a3169b05af57d06428fca23c33354f2d2] ci: enable automerge by default when backporting
git bisect bad 6f34f07a3169b05af57d06428fca23c33354f2d2
# good: [461ed5e6bd864ea5d5c37b86bc43728071af5d06] Merge pull request #30303 from neovim/backport-30300-to-release-0.10
git bisect good 461ed5e6bd864ea5d5c37b86bc43728071af5d06
# bad: [ef05c514aafb1eda4081f5f0408e93da0f7639dc] fix(runtime): add remaining missing commentstrings (#30252)
git bisect bad ef05c514aafb1eda4081f5f0408e93da0f7639dc
# bad: [e13564b9f44ec7b188c3010d2b3c4a3a3e405e6e] fix(lsp): handle encoding bounds in str_utfindex_enc
git bisect bad e13564b9f44ec7b188c3010d2b3c4a3a3e405e6e
# bad: [b55435f4386e9a138be7275919aa784f79a2fb03] fix(lsp): handle out-of-bounds character positions #30288
git bisect bad b55435f4386e9a138be7275919aa784f79a2fb03
# first bad commit: [b55435f4386e9a138be7275919aa784f79a2fb03] fix(lsp): handle out-of-bounds character positions #30288

eljamm · 2024-10-11T20:47:35Z

So this is neovim problem? Gotta open issue in their repo.

There is one already open in neovim/neovim#30675

The workaround in the issue above seems to be working as well. Just add offset_encoding = 'utf-8', to nixd's lspconfig:

NeoVim

local nvim_lsp = require 'lspconfig'
nvim_lsp.nixd.setup {
  offset_encoding = 'utf-8',
  settings = {
    ...
  },
}

NixVim

plugins.lsp.servers.nixd = {
  enable = true;
  extraOptions = {
    offset_encoding = "utf-8";
  };
};

GaetanLepage · 2024-10-13T15:32:13Z

Thanks for the investigation @eljamm !
Should we set offset_encoding = lib.mkDefault "utf-8"; or is this too opinionated (or simply not our concern) ?

Sk7Str1p3 · 2024-10-13T18:25:23Z

So I set offset_encoding = "utf-8" And I have no errors but syntax highlights changed a lot

Sk7Str1p3 · 2024-10-13T18:42:06Z

before VS after

eljamm · 2024-10-14T09:00:04Z

So I set offset_encoding = "utf-8" And I have no errors but syntax highlights changed a lot

I didn't notice at first as both colors are similar in my colorscheme. Interestingly, this also doesn't happen right after opening the file. To trigger it, I need to insert a newline under a line which has a glyph in it. Editing all the other lines is fine.

Should we set offset_encoding = lib.mkDefault "utf-8"; or is this too opinionated (or simply not our concern) ?

If it weren't for the broken syntax highlighting, I'd say we go for it, but perhaps it would be better to wait until we can patch this. I don't know how many people are affected, but they'll probably find this issue and the workaround and choose if it's worth to use it or just roll back Neovim.

MattSturgeon · 2024-10-14T09:24:42Z

Thanks for the investigation @eljamm ! Should we set offset_encoding = lib.mkDefault "utf-8"; or is this too opinionated (or simply not our concern) ?

Looking at the lua above, it seems this would need to be set in plugins.lsp.settings instead of plugins.lsp.servers.nixd.settings? While it isn't our default policy, I think I would personally like to set it as a default. At most, perhaps have a default-on option that does it.

So I set offset_encoding = "utf-8" and [...] syntax highlights changed a lot

before VS after

I can see differences between left & right, but I can't see any issues in your screenshot. At first glance it just looks like the right hand side is using LSP-based syntax highlighting while the left hand side is not.

Your left hand side also doesn't seem to be using treesitter highlighting,¹ which may explain why the difference is so stark?

plugins.treesitter.settings.highlight.enable ↩

stasjok · 2024-10-17T07:15:29Z

Looking at the lua above, it seems this would need to be set in plugins.lsp.settings instead of plugins.lsp.servers.nixd.settings? While it isn't our default policy, I think I would personally like to set it as a default. At most, perhaps have a default-on option that does it.

I don't recommend setting it globally, it can cause troubles in other servers. AFAIK content encoding is negotiated between the server and the client, but only since lsp spec 3.17 Before it was always utf-16.. And spec states that support for utf-16 is mandatory. By forcing it to utf-8 on neovim side encoding can be mismatched between the client and the server in some cases. This issue was a regression that was backported in 0.10. It should be fixed now in 0.10 branch, but I don't know when 0.10.3 will be released. Maybe a temporary patch can be included to nixpkgs: neovim/neovim@84bbbd9. Since it's the bug in Nvim, not in Nixvim, I don't think here is the place for workarounds line that, that are valuable only for one specific version of Neovim (0.10.2).

Byte index encoding bounds were previously only checked against the UTF-8 length, whereas the default is UTF-16, which caused undefined behaviour. neovim/neovim#30747 nix-community/nixvim#2390

Fix byte index encoding bounds, which were previously only checked against the UTF-8 length, whereas the default is UTF-16, which caused undefined behaviour. See: - neovim/neovim#30747 - nix-community/nixvim#2390

khaneliman · 2024-10-19T01:44:07Z

Have you tried using nightly neovim to see if it resolves your issues? I was able to use nixd with it (I had other plugins I needed to work so I rolled back to stable)

MattSturgeon · 2024-10-19T02:49:15Z

For anyone who missed it, there's also an overlay suggested here that you can use to apply the fix to neovim 0.10, while we wait for 0.10.3

  (final: prev: {
    neovim-unwrapped = prev.neovim-unwrapped.overrideAttrs (old: {
      patches = old.patches ++ [
        # Fix byte index encoding bounds.
        # - https://github.com/neovim/neovim/pull/30747
        # - https://github.com/nix-community/nixvim/issues/2390
        (final.fetchpatch {
          name = "fix-lsp-str_byteindex_enc-bounds-checking-30747.patch";
          url = "https://patch-diff.githubusercontent.com/raw/neovim/neovim/pull/30747.patch";
          hash = "sha256-2oNHUQozXKrHvKxt7R07T9YRIIx8W3gt8cVHLm2gYhg=";
        })
      ];
    });
  })

Issue: nix-community/nixvim#2390

Sk7Str1p3 added the bug Something isn't working label Oct 8, 2024

inclyc mentioned this issue Oct 9, 2024

NixVim \\ heavy flow of same errors then using nixd nix-community/nixd#604

Open

Sk7Str1p3 mentioned this issue Oct 11, 2024

LSP: NixD throws 'semantic tokens' errors after update to 0.10.2 neovim/neovim#30767

Closed

eljamm mentioned this issue Oct 17, 2024

neovim-unwrapped: fix byte index bounds checking NixOS/nixpkgs#349230

Closed

13 tasks

MattSturgeon mentioned this issue Oct 30, 2024

[HELP] nixd not working #2290

Closed

1 task

alesauce added a commit to alesauce/nixvim-flake that referenced this issue Nov 17, 2024

fix(config/lang/nix): Add utf-8 offset encoding for nixd

74b4481

Issue: nix-community/nixvim#2390

alesauce added a commit to alesauce/nixvim-flake that referenced this issue Nov 17, 2024

fix(config/lang/nix): Add utf-8 offset encoding for nixd

d498315

Issue: nix-community/nixvim#2390

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG] Heavy flow of errors with NixD #2390

[BUG] Heavy flow of errors with NixD #2390

Sk7Str1p3 commented Oct 8, 2024 •

edited

Loading

GaetanLepage commented Oct 8, 2024

inclyc commented Oct 9, 2024

Sk7Str1p3 commented Oct 9, 2024

khaneliman commented Oct 9, 2024

Sk7Str1p3 commented Oct 10, 2024

MattSturgeon commented Oct 10, 2024

Lassulus commented Oct 11, 2024

eljamm commented Oct 11, 2024

Sk7Str1p3 commented Oct 11, 2024

eljamm commented Oct 11, 2024

eljamm commented Oct 11, 2024 •

edited

Loading

GaetanLepage commented Oct 13, 2024

Sk7Str1p3 commented Oct 13, 2024

Sk7Str1p3 commented Oct 13, 2024

eljamm commented Oct 14, 2024

MattSturgeon commented Oct 14, 2024

stasjok commented Oct 17, 2024 •

edited

Loading

khaneliman commented Oct 19, 2024

MattSturgeon commented Oct 19, 2024 •

edited

Loading

[BUG] Heavy flow of errors with NixD #2390

[BUG] Heavy flow of errors with NixD #2390

Comments

Sk7Str1p3 commented Oct 8, 2024 • edited Loading

Description

Logs

Minimal, Reproducible Example (MRE)

GaetanLepage commented Oct 8, 2024

inclyc commented Oct 9, 2024

Sk7Str1p3 commented Oct 9, 2024

khaneliman commented Oct 9, 2024

Sk7Str1p3 commented Oct 10, 2024

MattSturgeon commented Oct 10, 2024

Lassulus commented Oct 11, 2024

eljamm commented Oct 11, 2024

Sk7Str1p3 commented Oct 11, 2024

eljamm commented Oct 11, 2024

eljamm commented Oct 11, 2024 • edited Loading

NeoVim

NixVim

GaetanLepage commented Oct 13, 2024

Sk7Str1p3 commented Oct 13, 2024

Sk7Str1p3 commented Oct 13, 2024

eljamm commented Oct 14, 2024

MattSturgeon commented Oct 14, 2024

Footnotes

stasjok commented Oct 17, 2024 • edited Loading

khaneliman commented Oct 19, 2024

MattSturgeon commented Oct 19, 2024 • edited Loading

Sk7Str1p3 commented Oct 8, 2024 •

edited

Loading

eljamm commented Oct 11, 2024 •

edited

Loading

stasjok commented Oct 17, 2024 •

edited

Loading

MattSturgeon commented Oct 19, 2024 •

edited

Loading