[RFC-16]: Add support for virtual microphone output

[lsamayoa]

Summary

Beginning of an RFC to add a Virtual Microphone output to OBS

Motivation

OBS is a powerful set of tools to manipulate live video streams that natively supports output to popular streaming services as well as rendering to a local video file. There are a huge number of people who engage in 1:1 or small-scale streaming using video conferencing software like Zoom or Google Hangouts. Many of these people have similar needs to those of traditional streamers, but for one reason or another cannot switch the video conferencing software they use (social graph, corporate policy). One way to deliver OBS output to these applications is to provide a virtual microphone in OBS.

View RFC

[lsamayoa]

@tobi @dodgepong a PR so we can work on "virtual microphone output" and get it to somewhere that is implemetable :)

[tobi]

thank you

[andrewodri]

Just by way of reference (as I really want to see this and the virtual camera functionality succeed)... BlackHole is a GPLv3 project that allows for the creation of macOS input/output devices; the code is clean and well documented.

I'd imagine that if streams could be provided in the an acceptable format, then building a bare bones PoC would likely be a matter of paring back UI and eliminating/stubbing unnecessary options (and of course developing the obs-plugin).

[alexmaasswca]

Any updates? This would be extremely useful for the Deaf and hard of hearing (so they can use it as a microphone for webcaptioner).

[bxm83]

I'd just like to throw my name in for this feature request. I am a teacher using OBS to livestream my teaching. Having this feature would make it a lot easier to monitor the videos I'm including in certain portions of my lessons.

[Acen]

Another library of note that could aid: https://github.com/ExistentialAudio/BlackHole

[ViperXL75]

I also use OBS for my webcam during my work, and I would like to see any updates on this addon.

[ViperXL75]

I can't help to wonder if funding will help to spike up the interest of the developers to create the virtual microphone output quicker.

[dodgepong]

Everyone, please try to restrict your comments on this RFC to that of productive conversation about the design and implementation plan of the RFC. Expressing support for the feature unfortunately does not make the feature come any faster. What makes it come faster is a solid design and implementation plan, which is what an RFC is supposed to be.

As I understand it from @jp9000, the biggest hurdle to this is that, due to the design of how video and audio are synced in an output, there will be some serious sync problems between the OBS virtual camera output and a theoretical virtual audio output, though I would have to defer to Jim to fill in the details. Basically, if we were to sync the OBS virtual camera with an audio output, it would introduce an unacceptable amount of latency to the virtual camera output such that it would become useless for most virtual camera use cases.

As for this RFC specifically, I have a few thoughts:

linux-puseaudio already implements pulseaudio output for linux which could be used from client applications already (maybe? no really checked the code yet)

A cursory glance at linux-pulseaudio suggests it only supports audio input, not output.

Add a button in the OBS UI below "Start Recording" and above "Studio Mode". In English localization, the button would be labeled "Start Virtual Microphone".

I'm not entirely sure this is a good idea. I suppose it makes sense to allow people to start/stop audio output from OBS but having a dedicated button for an audio-only output seems strange to me.

I suspect this RFC is largely copy/pasted from the virtual camera RFC and was never really fleshed out into a real proposal. I'm honestly leaning more toward closing this RFC and waiting for someone to come along and post a real design, unless @lsamayoa wants to weigh in on how they might attempt to address some of the technical challenges of this proposal.

[jthomerson]

Sorry I don't know enough about the underlying tech to know what's needed, but if it's helpful, I use ManyCam for this. There's a lot of things I like about OBS better, but if it helps in any way to look at what ManyCam is doing, I thought I'd mention it. We play videos (with sound) for our church every week through ManyCam and the audio is synced well enough that it's rarely ever distracting; I'd say it's never any more out of sync than what I've seen through, for example, Zoom screen sharing. This is on MacOS.

[therentabrain]

In terms of UI:

Under Settings - Audio - Advanced, add a checkbox for Enable virtual audio device

Below that there are six checkboxes for which stereo tracks to divert to the virtual audio device.

Or, if mixing channels is messy, then just a drop-down that lets you choose a single stereo track.

Perhaps right there also, a Latency setting, just like in Advanced Audio Properties, that allows you to manually sync if it's needed.

Perhaps these controls are disabled, if the virtual audio device is turned off.

I don't know how this would be accomplished on the backend, but I would love this feature, so my abilities and alpha testing are at y'all's disposal.

[tobyberesford]

I think the RFC is weak around "motivation". Clarifying why this is an essential feature would help define how it is solved.

For instance:

one reason or another cannot switch the video conferencing software they use

My use case is that I am giving a talk for Gamification Europe which is being hosted on remo.co. Clearly this is the platform for the whole conference so that is fixed. I want to switch between my video and talk to cut in prerecorded videos (MP4 media files) and play them with sound to my audience. I want to pipe everything to the remo.co audience down the camera/mic that Remo provides. Remo is using the browser so I assume WebRTC. I assume this is pretty similar to other virtual conference tech.

Since the whole reason I am using OBS is to be able to switch between cameras and media to give a more professional talk not being able to stream the audio through OBS at the same time is a bit of a deal breaker.

Hope this helps!

[bruxy70]

If this is what you miss, I can give plenty of examples. I thought it is kind of obvious. The virtual camera is great, but kind of useless without the audio. I can feed the actual mic directly to the video conferencing software - but that does not help with other multimedia sources played from OBS - like video files, audio files, web pages (including OBS.Ninjja) or NDI.
There is a workaround with feeding the audio monitor through Virtual Cable. But that is way too complicated, it prevents using monitoring for what it is meant for. And it is currently broken anyhow - it adds a delay to the sound. So OBS is currently not usable for this scenario, and that is a shame, given it is de-facto s standard.

[leoherzog]

Another use case: I'd love to be able to use the OBS Audio Mixer and Filters to have finer control over what audio I'm sending to virtual conferencing software. I'm using the excellent RNNoise filter on my mic to help suppress noise in OBS, but I can't easily capture the OBS audio output and direct it to Google Meet/Zoom/etc without fiddling with redirecting the monitor on every boot. If OBS was just constantly sending audio to a virtual mic, I'd be so happy.

[dodgepong]

Once again, this RFC is not a place to simply voice support or desire for the feature. The intent of an RFC is to discuss the technical design of the feature.

This is a feature that we already know people want. The RFC is the design stage. If you aren't posting comments on the design in the RFC, please refrain from posting your comment.

[leoherzog]

Sorry, I was responding to @tobyberesford's request for "clarifying why this is an essential feature would help define how it is solved."

[GitHub-ITGuy]

Experimenting with low-cost PoE security cameras (Speco) in a virtual teaching environment. Cameras are high latency so sync issues appear in Zoom meetings when paired with USB boundary microphones. Currently using Voicemeeter to add an audio delay but the maximum value seems to be 500ms. Suggest OBS virtual microphone include an audio latency feature with greater delay capability, possibly upwards of 2500ms.

[indolering]

@dodgepong: could you edit the top post to include something like,

Upvote and/or comment this feature here. Do not post non-technical comments on Github issue tickets!

---

On Linux, pulseaudio is being replaced with PipeWire (explainer, design doc, FAQ). PipeWire also supports Jack, PulseAudio, and ALSA APIs/ABIs. I wouldn't suggest using the compatibility layers, however: PipeWire was previously called PulseVideo until they ran into audio/video synchronization issues and decided to overhaul PulseAudio too. Using PipeWire directly would also mean that OBS works within the XDG permissions/sandboxing framework on Linux.

Edit: the Flatpak RFC also contains a bunch of information on PipeWire/JACK. Debian and Red Hat are all in on PipeWire, but it looks like you are going to have to support both JACK and PipeWire until at least Ubuntu 21.04 😞. From a maintenance perspective, I would prefer shipping this feature and just not supporting it on systems that lack PipeWire.

That being said, PipeWire only works on Linux. I found this ticket that summarizes both platform APIs and libraries that abstract over them. AFAIK, the only FOSS solutions on OS X for virtual audio are Blackhole and Soundflower.

WRT latency: just allow users to add latency to the audio stream. That's at least good enough when using Voicemeeter.

[lsamayoa]

Been busy at work, so I am just getting up to speed :)

I think the consensus is that the difficult part is not really implementing this type of functionality but to take the decision on how to implement it.

The main three problems that we need to solve:

• Audio/video sync - seems like adding a delay functionality is good enough for the first version.
• Virtual Hardware implementation
  • OSX: We already have created virtual devices on OSX and the HAL API is a lot more documented than the DAL API, so sound should not be difficult to implement (we have examples in BlackHole, and I already have a simple plugin that integrates with a modified version of BlackHole, not ready for github though).
  • Windows: I do not have any idea on how to implement an virtual audio device
  • Linux: It seems like most distributions already support PulseAudio or PipeWire to do this
  • General: Crazy idea, Maybe porting pipewire manager to Win and OSX to not maintain this as part of OBS ?
• IPC between OBS and Virtual Hardware
  • Windows: For IPC we have a very simple shared memory queue in order to transmit the messages from OBS and the virtual hardware (we could say it is an OBS specific protocol and implementation).
  • OSX: We use a Mach based protocol to send the messages to the virtual hardware from OBS (OBS specific too, and it is totally different with what we have on windows, even message structure)
  • Linux: We already have PipeWire
  • General: In general maintaining different IPC for different platforms IMHO is not scalable. Crazy idea, port PipeWire to OSX and Windows (at least the IPC protocol)? This would also allow us manage the video and audio in the daemon/virtual hardware manager.

With this said, maybe we need some direction from the maintainers on what would they see fit to move with.

Note: TBH I do not have much experience with desktop OSX or Windows apps development, my background is in Web development for linux but trying to learn as much as possible while contributing :)

[lsamayoa]

I suspect this RFC is largely copy/pasted from the virtual camera RFC and was never really fleshed out into a real proposal. I'm honestly leaning more toward closing this RFC and waiting for someone to come along and post a real design, unless @lsamayoa wants to weigh in on how they might attempt to address some of the technical challenges of this proposal.

Just to clarify, yes it is kind of a copy paste of the other RFC because in principle is the same thing. You can close this RFC if you really want, but I really think the discussion here is very valuable. I can update my RFC but like I stated in the comment above, it is up to the whole group (or the maintainers) to decided how to implement it first. (BTW I did a lot of investigation for the first RFC and wrote some parts of it)

[indolering]

After more research, I got a lot of this backwards.

WRT a virtual device, there is WASAPI for Windows, CoreAudio for OS X, ALSA for Linux, and OSS for the BSDs. There are virtual devices that support some combination of JACK, PulseAudio, PortAudio, and/or ASIO. But I can't find any single virtual device project that supports Linux, Windows, and Mac (let alone the BSDs).

PulseAudio, JACK, and PipeWire are not just libraries, but sound servers that handle things like resampling. PulseAudio's core design was compromised by early ALSA drivers and introduces a lot of latency. JACK was created as a low latency alternative, but is brittle and resource intensive. PulseVideo was created as a way to enforce access-control policies as part of the switch to Wayland. But PulseAudio's push based interface made it really hard to keep the audio and video stream in sync. So they started PipeWire, which handles all of the above and can dynamically trade latency for battery life.

However, there doesn't yet appear to be any virtual audio devices with a PipeWire interface. PipeWire provides JACK and PulseAudio compatible ABIs and it is possible to use JACK as a PipeWire backend. I don't know the downsides of these compatibility hacks, but @GeorgesStavracas might.

Maybe porting pipewire manager to Win and OSX to not maintain this as part of OBS ?

Gee, if only the Linux Desktop stack could double as a cross-platform toolkit ... 😭.

AFAICT, PipeWire defines a protocol to request access rights and enforce policy decisions within the server. Doling out privileges is left up to a session manager, like WirePlumber.

In general maintaining different IPC for different platforms IMHO is not scalable. Crazy idea, port PipeWire to OSX and Windows (at least the IPC protocol)?

The idea of porting PipeWire is not crazy: dbus, JACK, and PulseAudio have actively maintained ports to Windows, OS X, etc. But PipeWire has only recently become mature enough for production use on Linux and I couldn't find any mention of ports to other platforms.

Someone on IRC suggested I summon @wtay (core dev for GStreamer & PipeWire) to chime in here. He has a reputation for being extremely friendly and helpful!

TBH I do not have much experience with desktop OSX or Windows apps development, my background is in Web development for linux but trying to learn as much as possible while contributing :)

Audio APIs are very tricky to get right due to how fast audio real-time processing must be done and interactions between latency, buffering, and (re)sampling. I've wasted days trying to debug issues with Voicemeeter and I still have to manually restart the audio engine after a few hours (which is why I'm here :p). The PipeWire wiki has very helpful explanations for all of their high-level design decisions.

My advice would be to just focus on getting a single virtual device running smoothly on your platform of choice first. I'm assuming that this will evolve a lot like the virtual camera situation: plugins that add support for additional platforms over time.

[DDRBoxman]

Just posting a few tidbits to move the discussion along.

Here's Microsoft's WDM audio driver sample:
https://github.com/microsoft/Windows-driver-samples/blob/master/audio/sysvad/README.md

And a few projects that leverage it:
https://github.com/duncanthrax/scream
https://github.com/JannesP/AudioMirror

Things I'd like to see in the RFC about WDM if it makes sense as a technology choice:

• What level of code signing cert will we need?
• What's the install procedure?
• Technical implementation details?

I'd also like to see some technical specifics about implementing the macOS HAL device in the RFC.

An investigation should also be done into the OBS audio sync issues mentioned early on. It may not simply be an audio delay due to the way that OBS's audio buffer grows and syncs samples.