Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

qemu: Using -bios for firmware does not work with OVMF firmware #1231

Open
c3d opened this issue Jan 6, 2021 · 7 comments
Open

qemu: Using -bios for firmware does not work with OVMF firmware #1231

c3d opened this issue Jan 6, 2021 · 7 comments
Assignees
Labels
area/QEMU Issues specific to the qemu hypervisor bug Incorrect behaviour low-priority Low-priority issues

Comments

@c3d
Copy link
Member

c3d commented Jan 6, 2021

Get your issue reviewed faster

From code inspection, this applies to both the 1.x and 2.0 branches equally.

Description of problem

When using the firmware configuration option, the runtime translates that into a -bios command-line option for qemu. This will not work with an OVMF firmware, which also needs separate OVMF_CODE.fd for the read-only portion of the firmware and OVMF_VARS.fd for the associated variables.

Expected result

For an OVMF firmware, you'd expect the qemu command line to contain two entries, one for the read-only section of the firmware (OVMF_CODE.fd) and one for the read-write portion (OVMF_VARS.fd). This is typically done using:

 -drive if=pflash,format=raw,readonly,file=/path/to/OVMF_CODE.fd \
 -drive if=pflash,format=raw,file=/path/to/OVMF_VARS.fd \

See fuller story here, for example: https://wiki.debian.org/SecureBoot/VirtualMachine

Actual result

The current code generates the simpler -bios option that looks like:

-bios /path/to/OVMF_CODE.fd

In that case, qemu will most likely not know how to find the related OVMF_VARS.fd. This may or may not be important.

Further information

Example of command-line option with an OVMF firmware:

-name sandbox-csh -uuid a9ecf610-ebb5-42b4-bd8c-d72445f60601 -machine q35,accel=kvm,kernel_irqchip -cpu host,pmu=off -qmp unix:/run/vc/vm/csh/qmp.sock,server,nowait -m 2048M,slots=10,maxmem=129755M -device pci-bridge,bus=pcie.0,id=pci-bridge-0,chassis_nr=1,shpc=on,addr=2,romfile= -device virtio-serial-pci,disable-modern=false,id=serial0,romfile= -device virtconsole,chardev=charconsole0,id=console0 -chardev socket,id=charconsole0,path=/run/vc/vm/csh/console.sock,server,nowait -device virtio-scsi-pci,id=scsi0,disable-modern=false,romfile= -object rng-random,id=rng0,filename=/dev/urandom -device virtio-rng-pci,rng=rng0,romfile= -device vhost-vsock-pci,disable-modern=false,vhostfd=3,id=vsock-3113278101,guest-cid=3113278101,romfile= -device virtio-9p-pci,disable-modern=false,fsdev=extra-9p-kataShared,mount_tag=kataShared,romfile= -fsdev local,id=extra-9p-kataShared,path=/run/kata-containers/shared/sandboxes/csh/shared,security_model=none,multidevs=remap -rtc base=utc,driftfix=slew,clock=host -global kvm-pit.lost_tick_policy=discard -vga none -no-user-config -nodefaults -nographic --no-reboot -daemonize -object memory-backend-ram,id=dimm1,size=2048M -numa node,memdev=dimm1 -kernel /usr/share/kata-containers/vmlinuz-sev-es-5.6-v3.container -initrd /usr/share/kata-containers/kata-containers-initrd.img -append tsc=reliable no_timer_check rcupdate.rcu_expedited=1 i8042.direct=1 i8042.dumbkbd=1 i8042.nopnp=1 i8042.noaux=1 noreplace-smp reboot=k console=hvc0 console=hvc1 cryptomgr.notests net.ifnames=0 pci=lastbus=0 quiet panic=1 nr_cpus=240 scsi_mod.scan=none root=/dev/vda1 rootflags=data=ordered,errors=remount-ro agent.log=debug initcall_debug -bios /usr/local/share/qemu/OVMF_CODE.fd.kata -pidfile /run/vc/vm/csh/pid -smp 1,cores=1,threads=1,sockets=240,maxcpus=240
@c3d c3d added bug Incorrect behaviour needs-review Needs to be assessed by the team. labels Jan 6, 2021
@c3d c3d self-assigned this Jan 6, 2021
@jodh-intel jodh-intel added this to To do in Issue backlog Jan 6, 2021
@c3d c3d added the area/QEMU Issues specific to the qemu hypervisor label Jan 6, 2021
@c3d
Copy link
Member Author

c3d commented Jan 6, 2021

Some additional digging led me to believe that the -bios option actually may work well enough for our use case. Specifically, I tried the following manually:

 qemu-kvm -m 2048 -M q35,accel=kvm,kernel_irqchip  -bios /usr/share/OVMF/OVMF_CODE.fd -kernel /boot/vmlinuz -initrd /tmp/kata-test/initrd.img

and I got the kernel to boot reasonably far. This is not with secure boot enabled, though, so this may not work as intended if what are looking for is SecureBoot. But at least, qemu is smart enough to "do the right thing" when receiving both the -kernel and -bios options.

@c3d c3d added low-priority Low-priority issues and removed needs-review Needs to be assessed by the team. labels Jan 6, 2021
@c3d
Copy link
Member Author

c3d commented Jan 6, 2021

Setting "low priority" label since this seems to work somewhat OK with the current -bios option.

@c3d
Copy link
Member Author

c3d commented Jan 6, 2021

I was wrong about what the code generates. These seems to be a difference between 1.x and 2.0. According to a discussion with Sandeep Gupta on kata-dev, we have a -bios option for 2.0:

 -bios /usr/local/share/qemu/OVMF_CODE.fd.kata

But a partially correct -drive if=pflash option for 1.x:

 -drive if=pflash,format=raw,unit=0,file=/usr/local/share/qemu/OVMF_CODE.fd.kata,readonly

It's not very clear where that comes from in the govmm code:

func (config *Config) appendBios() {
	if config.Bios != "" {
		config.qemuParams = append(config.qemuParams, "-bios")
		config.qemuParams = append(config.qemuParams, config.Bios)
	}
}

I see nothing there that could produce the -drive option.

@jodh-intel
Copy link
Contributor

/cc @devimc, @sboeuf, @markdryan.

@c3d
Copy link
Member Author

c3d commented Jan 6, 2021

See thread discussing the topic.

Answer from Laszlo Ersek:

"-bios" should never be used with OVMF. If you don't need non-volatile
UEFI variables to persist from one boot of the domain to the next,
that's fine; in that case, please use a throw-away regular file for the
variable store pflash chip. this file should be instantiated from
"OVMF_VARS.fd", if you don't want the Secure Boot operational mode
enabled right off the bat; otherwise, please copy (create) the domain's
varstore from "OVMF_VARS.secboot.fd", before launching QEMU.

We have never tested SEV with "-bios" (well, because "-bios" is wrong to
use with OVMF in the first place, regardless of SEV); and yes there have
been SEV-related changes, specific to the flash driver in OVMF. (QEMU
too received SEV-related changes around the parts responsible for
loading OVMF, in "hw/i386/pc_sysfw.c".)

The particulars of a SEV failure, when using "-bios", aren't even worth
investigating -- please try to reproduce the issue first with two pflash
chips.

So pretty clear that we need to change the qemu command-line if we want to support safeboot and SEV.

@ariel-adam ariel-adam moved this from To do to area/CRI-O in Issue backlog Mar 9, 2021
@ariel-adam ariel-adam moved this from area/CRI-O to area QEMU/cloud-hypervisor in Issue backlog Mar 9, 2021
@c3d c3d added this to To do in Confidential containers May 11, 2021
@jimcadden
Copy link
Contributor

Ah, so I hit this problem in my SEV patches and worked around it in the follow way: In govmm I create the -drive if=pflash ... for OVMF at the same time as the -object sev-guest .... In kata-runtime, I specify the path to OVMF_CODE.fd in the config using firmware=, same as Bios, but I clear qemuConfig.Bios="" prior to govmm so that a -bios is never appended.

Not the most elegant, but it works with few code changes.

But, no support for OVMF_VARS.fd in this work-around solution.

@zvonkok
Copy link
Contributor

zvonkok commented May 16, 2022

Do we have any progress on this? Isn't this a requirement for confidential containers to load OVMF firmware? The other features that rely on this working are secure boot or measured boot. Running with the latest kata code the -bios is still used for the OVMF.

@jodh-intel

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/QEMU Issues specific to the qemu hypervisor bug Incorrect behaviour low-priority Low-priority issues
Projects
Issue backlog
  
area QEMU/cloud-hypervisor/firecracker
Development

No branches or pull requests

4 participants