Startup very slow when using fish as shell

[phgz]

Describe the bug

The use of this plugin drastically reduces the speed of opening and closing files in (n)vim. I use Kitty as the terminal emulator and was wondering if using, for example, Wezterm would improve that. Otherwise, what could/does cause that?

TMUX 3.3a;
Kitty 0.26.4;
NVIM v0.9.0-dev-124-gbd7ca10fd;

Using a profiler:

=====================================
Top 10 plugins slowing nvim's startup
=====================================
1       371.007   vim-tpipeline
2         1.597   gruvbox-theme
3         0.840   ultisnips
4         0.711   nvim-treesitter-textobjects
5         0.698   nvim-treesitter
6         0.436   denops.vim
7         0.418   telescope.nvim
8         0.377   nvim-treesitter-refactor
9         0.363   vim-fanfingtastic
10        0.351   indent-blankline.nvim
=====================================

To reproduce

No response

Expected behavior

No response

Terminal emulator

Kitty

Debug output

No response

[vimpostor]

1 371.007 vim-tpipeline

Oof, this is bad, is this milliseconds? With which command did you get this number?
I definitely can't reproduce this number, for me vim-tpipeline takes 0.59ms if I measure with vim-startuptime. With which plugin did you measure this?

The terminal emulator does not really have much impact, and even if it had, kitty is one of the faster terminals.

Most of this plugins time on startup is spent on spawning the asynchronous background tmux statusline updater process. Usually this should be fast, but it depends on your operating system.
For example on Linux, vim internally uses the fork() command before calling execve(), which means that the newly created process temporarily copies all memory pages from the vim process. While it is true that Linux's CoW mechanism mitigates this problem, Linux still has to copy all page tables, which can be quite large given that Linux has a page size of only 4kB. So if your vim on startup already uses 1GB of memory (e.g. from LSP plugins), then this will significantly slow down the fork() operation for creating vim-tpipeline's background process.

But of course this is just speculating in the dark, it is better if I profile this. Could you perhaps share your entire vim config (or at least the options set for this plugin), and your operating system, so that I can try to reproduce this?

P.S.: If you are using Windows, then process creation is very slow in general so I am not sure if we are able to fix this if you are on Windows.

[phgz]

Sorry, I forgot to specify it, but the OS is Ubuntu 20.04.

So the profiler I used is from hyiltiz/vim-plugins-profile, but I tried with yours and it gives similar result (372.82).

You can find my nvim config here. In particular, my statusbar is set to:

local function status_line()
  local status = " %{%get(b:,'gitsigns_status','')%}"
  status = status .. '%=%#GreyStatusLine#%{luaeval("NvimGPS()")}'
  status = status .. '%=%h%#RedStatusLine#%m%#BlueStatusLine#%r%#StatusLine# %F ▏%2c'
  return status
end
.
.
.
vim.o.statusline = status_line()

I tried to only set it to the last statement, (ignoring gitsigns and NvimGPS), but it is still the same.

When trying the plugin, I set the config for TMUX according to what you suggest in the README for these settings.

[vimpostor]

I tried your config in a Docker container, and I still cannot reproduce (vim-tpipeline takes 0.301 ms on startup with your exact config).

So it must be something else.
The easiest way to debug this, is if you start nvim like this:

nvim --cmd 'profile start profile.log' --cmd 'profile func *' --cmd 'profile file *tpipeline*'

Then quit nvim immediately after starting and upload the file profile.log here please.

With that info, I should be able to see what's going on with your setup.

[phgz]

Here it is: profile.log.

So from what I understand, it would be caused by tpipeline#util#set_size()?

Edit: It seems to also take time for tpipeline#cleanup(). The common point seems to be a call to tmux under the hood.

[vimpostor]

Yeah, so apparently calling system() calls are very slow on your computer.

What does vim output when you enter :let a = reltime() | echo system("true") | echo reltimestr(reltime(a))?
The result should be a very small number.
Do you get a (significantly) different number when you run :let a = reltime() | silent echo system("true") | echo reltimestr(reltime(a))?

It would be interesting to find out why they are slow on your computer.

But longterm I want to get rid of that last remaining blocking system() call on startup anway, so might as well find a workaround now (this one is gonna be a bit harder though to get rid off).

[phgz]

So I investigated further for 2 other things:

1. The executions were made on a SSH remote session, but I just tried it locally, and it did not reduce the time in a significant matter (profiler indicates 307.270 for vim-tpipeline).

2. My interactive shell is fish, but I guess the calls are made in a sh/bash subshell. I tried running from bash, but I had the same results.

The machine I ran the tests on has 64G RAM, so I hardly see a memory problem.

Finally, here is the output of

:let a = reltime() | echo system("true") | echo reltimestr(reltime(a)): 0.383812

and

:let a = reltime() | silent echo system("true") | echo reltimestr(reltime(a)): 0.375149
.

I wonder why these calls are so slow. Could it be due to the fact that I use an AppImage for the nvim executable?

Edit: I tried with a traditional binary and it is still the same.
Edit2: I tried the same calls directly with vim (versions 9.0.820 and 8.1.3741) and the results are similar.

[vimpostor]

Very weird indeed, some googling seems to confirm this is a common issue when using vim with fish:

• Startup of vim-fish is insanely slow dag/vim-fish#34
• https://www.reddit.com/r/neovim/comments/olrtof/a_fix_for_neovim_been_slow_for_fish_users/
• vim slowness (with enhanced slowness due to setting shell in vim) fish-shell/fish-shell#7004

Apparently the fix is to set the shell in vim to bash or sh:

" vimrc
set shell=/bin/bash
-- or init.lua
vim.opt.shell = "/bin/bash"

Does this fix the issue?

[phgz]

Indeed it does! It now drop to ~2.7 ms for the plugin (still due to those calls, but that is now in a reasonable range).

Thank you for taking the time to debug that with me even if it was not related to this particular plugin, although it is the first time that issue manifest itself despite having tried a lot of plugins.

[vimpostor]

Good to hear that it resolved the issue for you. :)

Still I think ~3ms is a little bit too much, I will see if I can get rid of that last remaining blocking call as well.

[jay-babu]

Can you please pin this issue or add it to the README? I almost removed this plugin because the latency it added haha though the functionality it provides is great!

[jay-babu]

Very weird indeed, some googling seems to confirm this is a common issue when using vim with fish:

• Startup of vim-fish is insanely slow dag/vim-fish#34
• https://www.reddit.com/r/neovim/comments/olrtof/a_fix_for_neovim_been_slow_for_fish_users/
• vim slowness (with enhanced slowness due to setting shell in vim) fish-shell/fish-shell#7004

Apparently the fix is to set the shell in vim to bash or sh:

" vimrc
set shell=/bin/bash
-- or init.lua
vim.opt.shell = "/bin/bash"

Does this fix the issue?

vim.opt.shell = vim.fn.exepath("bash") is a little more dynamic