Implement VirtIO sound device

[Cuda-Chen]

Currently, semu lacks of sound playing feature.

To implement, we can use VirtIO sound with ALSA architecture.

[jserv]

we can use VirtIO sound with ALSA architecture.

Can you illustrate the progress and the potential integration considerations?

[Cuda-Chen]

Hi @jserv ,

For the progress:

• The system can boot up and the ALSA driver will be registered (using make check but not sudo make check as it will complains incorrect parameter related errors).
• The host system can see the ALSA driver in system settings (named ALSA plug-in [semu] in Volume Levels).

For potential integration considerations:

• We should install the ALSA utilities such as alsa-utils for testing the sound device.
• TinyALSA can be used if semu is aiming for a lightweight emulator.

[Cuda-Chen]

Hi @jserv ,

For the supporting operations mentioned in #53, to let semu plays sound I consider it requires to support more operations (and the operations are mentioned in VirtIO official document), should we investigate then list the operations that have to be implemented to support the common sound operation (e.g., playing sound, querying sound device information, etc.)?

[jserv]

should we investigate then list the operations that have to be implemented to support the common sound operation (e.g., playing sound, querying sound device information, etc.)?

Yes, go ahead.

[Cuda-Chen]

For this issue, I am going to implement VirtIO sound device supporting these operations:

• querying the device information
  • VIRTIO_SND_R_PCM_INFO
  • VIRTIO_SND_R_CHMAP_INFO
  • VIRTIO_SND_R_JACK_INFO
• playing sound (PCM, specifically)
  • VIRTIO_SND_R_PCM_SET_PARAMS
  • VIRTIO_SND_R_PCM_PREPARE
  • VIRTIO_SND_R_PCM_RELEASE
  • VIRTIO_SND_R_PCM_START
  • VIRTIO_SND_R_PCM_STOP

[Cuda-Chen]

Update:

• Sound Card
  • After investigation, the VirtIO SoundCard at platform/f4400000.virtio/virtio2 is actually attached to the kernel.
  • However, I can only see controlCX (X stands for the card number of VirtIO SoundCard) in /dev/snd/.
  • Compared to LoopBack, which creates the following files in /dev/snd/:
    • controlCX
    • pcmCXDYc
    • pcmCXDYp
• PCM
  • As the VirtIO SoundCard does not expose any endpoint for PCM playback, there is no way to play sound via VirtIO SoundCard.

I am going to solve the sound card endpoint issue first.

[Cuda-Chen]

Update:

• Sound Card
  • The VirtIO Sound Card can be brought up after initialization.
  • I can see the pcmCXDYc appears in /dev/snd/, and aplay -l can list the VirtIO Sound Card.
• PCM
  • speaker-test and aplay exit with non-zero return value because an unknown type (0x0) is sent from driver to virtio-snd after pcm_prepare state.

[Cuda-Chen]

Statue update:

• PCM
  • After investigation, semu VirtIO device receive the data from TX queue after VIRTIO_SND_R_PCM_PREPARE state.
  • I am going to implement to receive the PCM frames from TX queue to buffer so that we can play the sound.

[jserv]

• I am going to implement to receive the PCM frames from TX queue to buffer so that we can play the sound.

Do you think whether if single-threaded queue manipulation is enough. I am not sure that such TX queue can be operated without extra threads.

[Cuda-Chen]

Do you think whether if single-threaded queue manipulation is enough. I am not sure that such TX queue can be operated without extra threads.

For my current findings, qemu and rust-vmm do not use any extra threads to operate TX queue.
However, we may consider using extra threads as it seems we have to notify the device to complete transmission once it gets PCM frame from TX queue.

[Cuda-Chen]

Statue update:

• PCM
  • Findings: the driver sends arbitrary number of PCM frames when transferring data.
  • I will implement a data structure which holds these arbitrary number of frames (e.g., queue or scatter list).

[Cuda-Chen]

Statue update:

• PCM
  • The queue holding arbitrary number of PCM frames is implemented.
  • The driver (guest Linux OS) still sends the PCM frames after the device receives the PCM frames. It seems that we have to send some signal to inform the driver that the device has received the PCM frames and it should send pcm_start to play the sound.

[Cuda-Chen]

Update:

• PCM
  • Create a dedicated thread to handle TX.
  • The pcm_release signal is sent asynchronously.

Actions:

• PCM
  • Create a lock in pcm_start/pcm_stop state.
  • Check the possible implementation to handle pcm_release state.

[jserv]

• PCM
  • Create a lock in pcm_start/pcm_stop state.
  • Check the possible implementation to handle pcm_release state.

@idoleat, can you comment the recent work of #53 ?

[Cuda-Chen]

• PCM

  • Create a lock in pcm_start/pcm_stop state.
  • Check the possible implementation to handle pcm_release state.

@idoleat, can you comment the recent work of #53 ?

Hi @jserv ,

I thought you may tag the wrong person.
So should I conclude the recent work of #53?

[jserv]

I thought you may tag the wrong person.

No, I was intentionally referring to him.

should I conclude the recent work of #53?

Go ahead.

[Cuda-Chen]

should I conclude the recent work of #53?

Go ahead.

Hi @jserv , I am going to summarize the recent work in this comment.

Past

• Initialize VirtIO sound device.
• Implement a thread for receiving PCM frames sent from driver.

Ongoing

• Implement pcm_release state.
• Implement a ring buffer to store the PCM frames for playback.