Skip to content
Oliver Caldwell edited this page May 2, 2022 · 17 revisions

Work in progress!

Conjure adds support for various languages through "clients", these are Lua modules that expose a selection of functions and values that allow the wider Conjure UX to interact with the target language. There’s more information on configuring Conjure to point at clients depending on the filetype under :h conjure-clients.

Some of these functions and values are optional so the support for various features in the Conjure client interface varies from client to client. Sometimes this is due to time constraints (there’s only so many hours to work across many languages and ecosystems), other times it’s due to the way we’re forced to communicate with a REPL holding us back feature-wise.

There are also some features that aren’t strictly part of the interface and may vary in vague ways language to language (such as prompting for user input if some sort of "read line" function is invoked). In the table and documentation below you will find the following:

  • All supported language clients and their related information

  • Supported features per language client

  • Caveats around specific language clients and their feature support

  • Details about each feature, it’s implementation and how to pursue implementing it for a client

This table is dense and tries to bring together a lot of disparate information spread out across a lot of code. It will fall out of sync with the released code from time to time, if you spot an issue, please help the community by correcting it.

TODO

  • ❏ Data table listing the languages, features and their overlap

  • ❏ Information about each client (source code, docs etc)

  • ❏ Information about each feature (how to implement it, examples, what it does)

  • ❏ Caveats about specific client / feature implementations such as shortcomings due to technical limitations in a specific REPL

Client Config State [Context] Eval Eval file Input prompt Doc lookup Go to def Completions Form node Debugger

Clojure (nREPL) src

Definitions

Client

A Lua module that can be pointed to by the g:conjure#filetype#…​ configuration value. It must export various functions and values that conform to the Conjure client interfaces as described in the :h conjure-clients section of the help text. You can find details of each of the possible supported features in the rest of this document.

All you really need to know is that a client is a Lua module that exports some known values, some optional, some required. There’s nothing special about them other than that, they can be stored inside Conjure’s main repository or in a 3rd party plugin that you configure your Conjure setup to load for a specific file type.

They’re lazily loaded as you enter a supported filetype, so there’s no harm in having many of them laying around ready to go.

Config

The client calls out to conjure.config/merge to merge their defaults into the shared tree at module load time. This allows the user to override configuration through the g:* global values as they see fit. The config should be nested under {:client {:name {:transport …​}}}, so Clojure’s looks like this:

(config.merge
  {:client
   {:clojure
    {:nrepl
     {:foo true}}}})

State

Clients can store stateful values inside Conjure’s state system. The benefit for storing stateful things inside this system is that user’s can set a state key with :ConjureClientState any-distinct-string-here! and get a whole new parallel copy of all the client state.

For Clojure, this includes nREPL connections, so you can actually set up an autocmd to set your :ConjureClientState …​ key to clj or cljs as you hop around your project’s front and backend source code (for example). Clojure’s usage looks like this:

(defonce get
  (client.new-state
    (fn []
      {:conn nil
       :auto-repl-proc nil
       :join-next {:key nil}})))

Then the rest of the Clojure client calls this get function with various path parts to extract the current value depending on the state key (which defaults to "default") like so: (state.get :conn) or (state.get :foo :bar :baz) to drill into something.

So calling new-state returns a lookup function that sort of behaves like get-in but with varargs. You give new-state a function so that it can be called for you every time a new state is created. If a state key is reused you get the original state value back out. https://github.com/Olical/conjure/blob/842c81892648de759e639ad2d395757b98be06d5/fnl/conjure/client/clojure/nrepl/server.fnl#L58-L90 === Context

The term used to describe the "name" of the buffer in terms of the connected REPL or language. So for Clojure this is the namespace name, for Fennel+Aniseed this is the module name. Both of these roughly line up with the file path but they sometimes have subtle differences.

Most REPLs and languages have some concept of this and knowing this value allows us to hop the REPL over into the correct context as we change buffers or evaluate code. For Clojure, we send the namespace name along with every nREPL operation to make sure every evaluation happens in the correct namespace context.

Conjure will either use the context-pattern (a Lua pattern) or context function exported by the client module to determine the context for the file. In the pattern case, it’ll use the result of the first capture group, with the function call it uses the returned value. The function will be given the first few lines of the file and your job is to parse the context out of that (if any).

This context string extracted from the buffer is then passed to any evaluation related client function calls under opts.context for you to use when running code or looking things up.

Depending on your language the pattern may be enough to get by or you may need a function with a few steps of breaking the string up until you can reasonably determine the context string. This varies from client to client but Clojure’s context function looks like this:

(defn context [header]
  (-?> header
       (parse.strip-meta)
       (parse.strip-comments)
       (string.match "%(%s*ns%s+([^)]*)")
       (str.split "%s+")
       (a.first)))

Eval

The eval-str method is invoked on clients from various parts of the Conjure UX. This includes evaluation of forms (through tree-sitter or older methods), :ConjureEval invocations, visual selections, eval at mark etc. You don’t really need to know where it came from or why, you just need to evaluate the string under opts.code (opts being the single table argument your function receives) using the various other options as you see fit.

You can see a detailed breakdown of the values under opts in the helptext with :helpgrep eval-str. The best way to really understand this and implement your own is to read through the Clojure implementation which calls out to other functions in this relatively complex client.

Eval file

Just like eval-str except you’re given a path and told to evaluate that somehow. I think the Fennel Aniseed client is a good example of this one, it actually reads the file into memory and then delegates to eval-str!

This demonstrates how the opts table is fairly similar across the evaluation functions. The contents of opts could changes over time (hopefully only growing), so I’ll still refer to :helpgrep eval-file for further details.

Input prompt

This isn’t a client module feature but a function you can rely on if your client supports prompting for user input for whatever reason.

The nREPL transport implementation (used by the Clojure nREPL client, but not exclusive to it) will detect when input is required thanks to the nREPL protocol and then prompt the user on your behalf.

This feature support relies on conjure.extract/prompt to receive input but it’s up to you and your REPL to know when that’s required and how it gets there.

Doc lookup

Much like an evaluation, you’re given a string and your client has to turn that into documentation…​ somehow. You must implement the doc-str function which, like eval-str and eval-file takes some opts and does whatever it sees fit.

In the case of Aniseed’s client it wraps the opts.code in a documentation lookup string and evaluates it. With Clojure and nREPL we can send a (doc …​) chunk of code or invoke nREPL’s info operation.

Go to def

Once again, a similar story! Your doc-str function is given some opts which contains a code string and you have to work out the rest. Clojure’s implementation is probably the most extensive here too. It handles a bunch of fallback cases since Clojure can be referring to other Clojure code local to your project, in a library or even Java!

Completions

Your client module needs to contain a completions function for this to work, it’ll be given another opts table with a prefix string inside it, that’s the prefix you’re trying to find and list results for. The results you’re returning should fit the format of any Neovim omnicompletion function which is well documented and standardise.

There’s nothing special here, it’s a normal omnicompletion function except you’re called automatically when required, possibly part of an async completion system (all managed for you!) and you’re given more context to work with. So a bunch of the hard steps are done for you, you just have to turn a prefix string into a list of omnicompletion compatible results. Conjure handles the rest of the plumbing.

The Aniseed client is a good example of this system despite it’s complexities. It’s doing a lot of work to evaluate chunks of code to attempt to pull values out of the local scope and peek into tables to get their keys but I think it’ll show you everything that needs to be done.

Form node

A mostly unused feature for now, it’s called from the conjure.tree-sitter module as it tries to find a good chunk of code to extract (presumably for evaluation). So as it walks up the tree it’ll repeatedly ask your client’s form-node? function if the given tree-sitter node is a "form" or not, essentially meaning "can you evaluate this or should I walk up higher?".

This allows each client to decide which kind of nodes it considers runnable or not. You might want to disregard the if part of if (a) { b() } for example. This concept should make it easier for client implementers to support non-Lisp languages which sometimes don’t have clearly defined form boundaries.

Debugger

This doesn’t exist yet but hopefully will do soon. It’ll involve another set of functions your client must implement to participate. The actual debugger UI will be provided by another 3rd party plugin the user will have to install and learn but Conjure will be the bridge between your language and that debugger UI plugin.

This will probably only be possible for a small subset of clients that have very rich REPL toolchains, such as Clojure and nREPL+CIDER (which is the initial target for this concept).