CLAP_PARAM_IS_AUDIO_RATE flag for parameters supporting audio rate automation/modulation

[robbert-vdh]

I realized that there's currently no way to tell which parameters support audio rate automation/modulation and which do not. Knowing this in advance would be very useful, because it allows the host to send accurate automation values per-sample for parameters that support it, and only one value per buffer for parameters that don't. Currently Bitwig for instance sends only one parameter update every 64 samples when using the modulators, presumably because very few plugins actually use it (JUCE doesn't sample accurate automation at all in the currently released version) and it would be a waste otherwise. But if the host does know that the plugin can use it, then it can send high resolution automation without wasting any resources. Being able to use Bitwig's Audio Rate modulator with plugins for instance would open up a huge world of new possibilities.

[abique]

What about CLAP_PARAM_IS_BLOCK_BASED maybe?

Because either the plugin accepts multiple values per process call or just one, like vst2 would.
I think audio_rate is a term that I'd like to reserve to 1 value for each samples. Basically it would allow the host to provide an audio buffer for the param automation.

I find the feature interesting but it leads to other questions.
Would it be interesting for the plugin to query a suggested smoothing time in samples from the host? Or for the host to query by how many samples the parameters are being smoothed?

Maybe the plugin wants to smooth differently one param versus an other or depending on the is_live flag?

Let's go back to IS_BLOCK_BASED. Usually in VST2 we set the parameter to its value at the end of the block, because of the plugin smoothing. Here that'd be problematic with clap maybe because you'd have the param value at the end of the event queue, which would not be convenient.

I think that this topic opens a lot of questions, leads to a more complicated code to maintain for the host while I'm not sure that we'd be able to really measure a performance win.

What do you think?

[robbert-vdh]

I feel like most plugins and parameters are block based, so having that be the default and having flags to indicate that a parameter can indeed make use of more than one value per block would make more sense to me. So opt-in instead of opt-out. In a sense that's equivalent to const being optional for locals in C++: the absence of it doesn't mean that the variable is not going to change. The programmer can just have forgotten about it. Similarly with CLAP_PARAM_IS_BLOCK_BASED, it not being there does not necessarily mean that the parameter can make use of more than one value per block. And since preparing and handling more than one parameter change event for a parameter per block is more work for both the host and the plugin, I feel like it would make much more sense to have a flag that indicates that the plugin can make use of that extra granularity.

Audio rate might indeed be the wrong term, but if a plugin can handle sample accurate automation, then unless it processes audio in very large internal blocks it should be able to use any granularity of parameter change events. CLAP_PARAM_SUPPORTS_AUDIO_RATE would make more sense (and that was the name I was originally going to propose), but every other flag uses _IS_ so I went with that instead.

Would it be interesting for the plugin to query a suggested smoothing time in samples from the host?

I don't think this is necessary. The correct smoothing time for a parameter very much depends on the parameter itself. You can for instance use much lower smoothing times for a gain parameter than you can for a filter cutoff parameter that gets used for calculating new IIR filter coefficients. Only the plugin knows what the best smoothing time for a parameter is.

Let's go back to IS_BLOCK_BASED. Usually in VST2 we set the parameter to its value at the end of the block, because of the plugin smoothing. Here that'd be problematic with clap maybe because you'd have the param value at the end of the event queue, which would not be convenient.

That's an interesting one actually. I don't think you can really encode it in the plugin standard, but it would definitely be a good idea to document that hosts should add all parameter change events for parameters that don't support sample accurate automation at the start of the queue.

[abique]

This issue would be fixed by #86 right?

[robbert-vdh]

Partially! That would allow true audio rate automation which is great, but it doesn't allow you to influence the rate at which the host sends mid-buffer automation/modulation for 'normal' automation/modulation events or disable them entirely. Because for most uses you probably don't want to use the raster approach, as that takes you back to square 1. The problem with VST3's sample accurate automation approach that CLAP solves is that in VST3 every parameter has its own automation queue, and the plugin then first needs to merge all of those queues together with the other events into one larger queue before it can do anything.

Being able to do audio rate modulation for single parameters like this would probably still be useful though. In those cases you may be able to split your processing loop up into into a normal loop with interleaved events and then a dedicated loops for the dense raster buffers.

[dromer]

Could this in practice be similar to LV2 CVPort? https://lv2plug.in/ns/lv2core#CVPort
I suppose audio rate parameter automation/modulation is intermittent, whereas CV is continuous?

Within LV2 such a port has an AudioPort formatted buffer, but then with extra control attributes like min/max/default and meaningful values. Which the Host can then route to other plugins, or to dedicated physical CV ports to interact with external gear (through DC-coupled i/o, for instance via JACK https://jackaudio.org/metadata/signal-type/).

Compared to VST3 CV spec which is just a plain audio port with an extra flag (so only normalized values possible).

Or, because CV concerns continuous frame processing, this would require a completely different API within CLAP?
Since CLAP parameters are polyphonic that also poses some issues for a "CV" implementation, although perhaps it would look more like how VCV implements polyphonic cables internally?

(sorry for the many - possibly completely unrelated to this topic - examples of audio rate controls but I'm trying to understand the different use cases here and what the overlap in functionality is)

[robbert-vdh]

@dromer That already exists: https://github.com/free-audio/clap/blob/main/include/clap/ext/draft/cv.h

Problem with CV is that it's completely separate from parameter automation. If you have n parameters that you want to support both regular event-based automation and CV automation for (let's just forget about modulation for a bit, that would add another layer of complexity here), then you end up with 2^n potentially different paths you need to take. That CV signal is also not tightly coupled to the parameter. There was a brief extension proposal for having something similar to CV signals in the form of dense sample buffers that would be sent for a parameter as part of an event, but then you're of course still in a similar situation and CLAP's simple event automation model suddenly becomes a lot more complicated. Which means most plugins likely wouldn't even use implement this extension. So that was put on hold until after CLAP 1.0.

[dromer]

Ah nice. Thnx for the link! Still seems quite preliminary and incomplete for what I think such ports should be able to do.

Here I was thinking more from practical reasons, but parameters are certainly different from CV ports.

Sorry for the tangent!

[baconpaul]

Also cv is no really polyphonic and parameter modulation is, which we will need to grapple with when we get to true audio rate stuff

[dromer]

Yes, understood. It's audiorate i/o from a different "angle", say.