[Feature] new history backend 'rqlite' implmentation

[vcup]

Hello theres.
This PR make Reedline and Nushell can storage command history into Rqlite, a distributed SQLite database. details here.
The new RqliteBackedHistory can connect to an Rqlite instance and storage history entries like sqlite.
Support for nushell will be another PR, commits on vcup/nushell#feat/rqlite.

Motivation / Benefits

I have multiple machines and some "shell snippet" can exec across them. Using Rqlite can sync command history across multiple machines. Heres some tricks to make history commands completion more handy by adding a tag before command like this

❯ #shorthand
<some command here>

Syncing history across multiple machines can greatly enhance this trick.

Also:

• Search all history whenever they are typed on which machines. also enhanced the trick: leaving tags after commands makes them easier to search.

• When Nushell's history file_format is "sqlite", it stores "hostname" into database. maybe we can use it to get nushell more intelligent? idk, but if we already save it, then synching history over multiple machines is natural.

I have tried use some file synching application to sync my history files. However, if I open multiple instance of Nushell in difference machine at same time, syncing will conflict because sqlite's auxiliary files(they also host-specific). Not syncing them is not a option because network will randomlly lags.

Thanks for considering this PR. Please let me know what you think.

[fdncred]

Thanks for trying to make reedline & nushell better. I'm guessing this won't move forward because it's kind of what atuin already does and atuin supports nushell. It's interesting but I'm not sure other team members will want to add this. We can wait and see if they respond.

We typically recommend that people join our Discord and discuss changes like this before implementing them in order to save them time.

[vcup]

Thanks for fast reply, I have tried atuin before, I thinks is has few issues below:

• Syncing is not realtime, it sync every 10min by default. That's quite slowing than realtime sync on rqlite.
• Not native enough, we can't seen history entry by built-in history command.
• Also, synchronized history entry from remote server are not availble for history completion.
• Some weird behavior, press up arrow to selecting history entry will also credited duration on prompt. (on starships, not sure for now)

However, I also knew this solution has some defect:

• Safety deploy Rqlite and access them across multiple machines, which mean across NAT. this usually need to setup an VPN.
• Hard to setup safety Rqlite network, the Raft port need protect if they running on public network, which mean users need to configure firewalls.
• We can't provide any help more than write some tutorials to help users setup they Rqlite instance.

Anyway, I like this implementation, if have someone want to try it, I believe they can learn many things from this. (e.g. raft protocol)

[devyn]

I'll also add that it's generally not a good idea to run raft over the public internet anyway, because it is somewhat latency sensitive and is fully consistent rather than available, so you'll struggle to use it if you or some of your other machines lose network access.

Raft is really meant for use within a local intranet, especially a single cluster of datacenters, to provide redundancy on that scale. It's not really intended for you to link up all of your machines across the web, even if you were to do it well (perhaps doing Wireguard peer-to-peer)

A centrally hosted service where you keep a local copy of your history data and it gets synchronized (like how git works) is much better for that reason

If atuin is not tightly integrated enough, we should try to provide ways for it to do so with the plugin API

[vcup]

I agree at this point, but is not necessaily to run raft over public internet, We can just expose http api and config some password for it (VPN also a option for me).
But I also think on this scenario, network got some lag is acceptable, not matter 1sec or 2sec late on synching history. (that is already huge enough and rarely)

However, I thinks i can setup some experimental server, use traffic controll incressing latency and observe this PR is it work well.

[vcup]

But I also think on this scenario, network got some lag is acceptable, not matter 1sec or 2sec late on synching history.

I was wrong, I just make a quick try, even latency distribute in 140ms ~ 170ms on 6 nodes rqlite network, entering command also consuming 500ms at least, fatally is it occurred on every command executing.
maybe this PR can only use rqlite as a networked SQLite.

[fdncred]

I don't think we're ready to accept this. Thanks for all the effort.