VMPL evaluation

[mmisono]

• Micro benchmark
  • #VC latency
• SVSM performance
  • SVSM https://github.com/AMDESE/linux-svsm
  • coconut https://github.com/coconut-svsm/svsm
  • vTPM https://github.com/svsm-vtpm/linux-svsm/blob/svsm-vtpm-preview/README-vtpm.md

[mmisono]

COCONUT-SVSM has its own versions software stack (cf. https://github.com/coconut-svsm/svsm/blob/main/INSTALL.md)

• COCONUT-SVSM: https://github.com/coconut-svsm/svsm
• QEMU: https://github.com/coconut-svsm/qemu
• OVMF: https://github.com/coconut-svsm/edk2.git
• Linux (guest/host?): https://github.com/coconut-svsm/linux/tree/svsm

AMD version

• SVSM: https://github.com/AMDESE/linux-svsm
• QEMU: https://github.com/AMDESE/qemu/tree/svsm-preview-v3
• OVMF: https://github.com/AMDESE/ovmf/tree/svsm-preview-v3
• Linux (guest): https://github.com/AMDESE/linux/tree/svsm-preview-guest-v3
• Linux (host): https://github.com/AMDESE/linux/tree/svsm-preview-hv-v3

[Sabanic-P]

Current Problem:

[Variable]: Rewritten a preexisting variable(0x00000006) with different attributes(0x00000007) - 605DAB50-E046-4300-ABB6-3DD810DD8B23:MokListTrustedRT
InstallProtocolInterface: 605DAB50-E046-4300-ABB6-3DD810DD8B23 7D58AF00
FSOpen: Open '\EFI\ubuntu\grubx64.efi' Success
PixelBlueGreenRedReserved8BitPerColor
ConvertPages: range 1000000 - 5266FFF covers multiple entries
ConvertPages: range 1000000 - 5266FFF covers multiple entries
CpuDxe: 5-Level Paging = 0
MpInitChangeApLoopCallback() done!
SetUefiImageMemoryAttributes - 0x000000007F6E4000 - 0x0000000000009000 (0x0000000000000008)
SetUefiImageMemoryAttributes - 0x000000007F6DD000 - 0x0000000000007000 (0x0000000000000008)
SetUefiImageMemoryAttributes - 0x000000007F6D7000 - 0x0000000000006000 (0x0000000000000008)
SetUefiImageMemoryAttributes - 0x000000007F6D0000 - 0x0000000000007000 (0x0000000000000008)
SetUefiImageMemoryAttributes - 0x000000007F6C0000 - 0x0000000000010000 (0x0000000000000008)
SetUefiImageMemoryAttributes - 0x000000007F6BB000 - 0x0000000000005000 (0x0000000000000008)
SetUefiImageMemoryAttributes - 0x000000007F6B4000 - 0x0000000000007000 (0x0000000000000008)
SetUefiImageMemoryAttributes - 0x000000007F6B0000 - 0x0000000000004000 (0x0000000000000008)
SetUefiImageMemoryAttributes - 0x000000007F6AB000 - 0x0000000000005000 (0x0000000000000008)
KVM: unknown exit reason 24
EAX=0000001a EBX=00000073 ECX=00000000 EDX=00000000
ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000
EIP=00000000 EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0000 00000000 00000000 00000000
CS =0000 00000000 00000000 00000000
SS =0000 00000000 00000000 00000000
DS =0000 00000000 00000000 00000000 
FS =0000 00000000 00000000 00000000
GS =0000 00000000 00000000 00000000
LDT=0000 00000000 00000000 00000000
TR =0000 00000000 00000000 00000000
GDT=     00000000 00000000
IDT=     00000000 00000000
CR0=80010033 CR2=00000000 CR3=00000000 CR4=00000668
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000
DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000900
Code=<??> ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??

Qemu (coconut version) is able to start COCONUT-SVSM which can successfully load the firmware but in the end fails at and KVM error.
The following command was used:

qemu-system-x86_64 \
-enable-kvm \
-cpu EPYC-v4 \
-machine q35,confidential-guest-support=sev0,memory-backend=ram1,kvm-type=protected \
-object memory-backend-memfd-private,id=ram1,size=8G,share=true \
-object sev-snp-guest,id=sev0,cbitpos=51,reduced-phys-bits=1,svsm=on \
-smp 8 \
-no-reboot \
-drive if=pflash,format=raw,unit=0,file=firmware/OVMF_CODE.fd,readonly=on \
-drive if=pflash,format=raw,unit=1,file=firmware/OVMF_VARS.fd,snapshot=on \
-drive if=pflash,format=raw,unit=2,file=svsm/svsm.bin,readonly=on \
-drive file=guest.qcow2,if=none,id=disk0,format=qcow2,snapshot=off \
-device virtio-scsi-pci,id=scsi0,disable-legacy=on,iommu_platform=on \
-device scsi-hd,drive=disk0,bootindex=0 \
-vga std \
-serial stdio \
-serial pty

The guest OS uses the following kernel: https://github.com/coconut-svsm/linux/tree/svsm

[Sabanic-P]

SEV-SNP works without SVSM using this command:

qemu-system-x86_64 \
  -cpu EPYC-v4,host-phys-bits=true \
  -smp 4 \
  -m 8G \
  -machine q35,memory-backend=ram1,confidential-guest-support=sev0,kvm-type=protected,vmport=off \
  -object sev-snp-guest,id=sev0,cbitpos=51,reduced-phys-bits=1,init-flags=0,host-data=b2l3bmNvd3FuY21wbXA \
  -object memory-backend-memfd-private,id=ram1,size=16G,share=true \
  -enable-kvm \
  -nographic \
  -blockdev qcow2,node-name=q2,file.driver=file,file.filename=guest.qcow2 \
  -device virtio-blk-pci,drive=q2,bootindex=0 \
  -netdev user,id=net0,hostfwd=tcp::2223-:22 \
  -device virtio-net-pci,netdev=net0 \
  -drive if=pflash,format=raw,unit=0,file=firmware/OVMF_CODE.fd,readonly=on \
  -drive if=pflash,format=raw,unit=1,file=firmware/OVMF_VARS.fd

[mmisono]

@Sabanic-P You can also try the latest AMD version

[Sabanic-P]

With both the older version of coconut and the AMD version of the kernel, it successfully boots and runs in VMPL2.

[    1.355371] Memory Encryption Features active: AMD SEV SEV-ES SEV-SNP
[    1.663432] SEV: SNP running at VMPL2.
[    1.664691] SEV: Using SNP CPUID table, 31 entries present.
[    2.276424] SEV: SNP guest platform device initialized.
[    5.870195] sev-guest sev-guest: Initialized SEV guest driver (using vmpck_id 2)

[mmisono]

How old is it? For the moment I think we can use that version but you can also create an issue at the coconut-svsm repo

[Sabanic-P]

It svsm-guest branch. The last update was 9 month ago. The newer svsm branch should definitely support non-VMPL0 execution based on the latest commit.

[mmisono]

There is #VC handler in coconut-svsm coconut-svsm/svsm#55
As a first step, let's try to invoke a #VC handler somehow in a controlled manner.

[mmisono]

TODO

• Check how to get log (or output log) from SVSM module
• Cause #VC (cpuid or io instruction?) and see if we can log the message
• Modify #VC handler

[Sabanic-P]

The Guest running in VMPL2 can communicate with the SVSM via shared memory (svsm_caa) and a hypercall.
SVSM assigned every vcpu a 4k page. The protocol itself only requires 8B.
The call is identified via the rax register.

Once the svsm is in control, it is able to log again.

TODO

• Extend SVSM to change VMPL levels
• Add ways to change execution to other VMPL level

[mmisono]

@Sabanic-P
Can you write how to reproduce it (how to try it)? You can create a repository if you have some code (or you can add code in this repo)