no bootable volumes (basedisk, diffdisk, ...)

[till]

Hi,

I'm trying to boot Flatcar Linux image with lima. I started off with the openstack image (stable channel). Log-wise it'll never complete the ssh check, when I start it without -display none, I see it cannot find a bootable disk.

I've dug into the code a bit and I am stuck at the logic of where it determines if the disk is a CDROM (ISO9660) and then attempts to boot from the diffdisk it created. I am not sure what that check really "means", or why that is necessary for it to complete. I tinkered with it a bit and when I set disk: 0, I effectively disable the diffdisk but then it doesn't seem to find a boot sector on the raw image either.

It seems the diffdisk is supposed to be backed by the basedisk. But that part somehow doesn't work as expected. When the machine boots, it'll revert to PXE booting as no disk is bootable. The diffdisk is also really small, even though I supplied disk: "10GiB". I am pretty sure I am doing this wrong.

Part of me thinks an additional disk would be easier than the diffdisk magic? But I don't understand why, I looked around in tickets, source code, discussions. If someone has more context, much appreciated.

LBNL, I tried booting a qemu image from Flatcar as well, same result. Even though it (sort of) works without lima. I dumped a few things from lima and assembled the qemu call.

base=$(pwd)/.base
rm -rf $base
mkdir -p $base

disk=/path/to/flatcar_production_qemu_image.img

/usr/local/bin/qemu-system-x86_64 -m 1024 \
    -cpu host \
    -machine q35,accel=hvf \
    -smp 1,sockets=1,cores=1,threads=1 \
    -boot order=c,splash-time=0,menu=on \
    -drive if=virtio,format=qcow2,file=${disk} \
    -drive if=virtio,format=raw,readonly=on,file=/Users/till/.lima/flatcar/cidata.iso \
    -netdev user,id=net0,net=192.168.5.0/24,dhcpstart=192.168.5.15,hostfwd=tcp:127.0.0.1:64619-:22 \
    -device virtio-net-pci,netdev=net0,mac=52:55:55:cd:fb:c7 \
    -device virtio-rng-pci \
    -device virtio-vga \
    -device virtio-keyboard-pci \
    -device virtio-mouse-pci \
    -device qemu-xhci,id=usb-bus \
    -parallel none \
    -chardev socket,id=char-serial,path=$base/serial.sock,server=on,wait=off,logfile=$base/serial.log \
    -serial chardev:char-serial -chardev socket,id=char-qmp,path=$base/qmp.sock,server=on,wait=off \
    -qmp chardev:char-qmp \
    -name lima-flatcar \
    -pidfile $base/qemu.pid

By "sort of", I mean, it boots - not without issues (I think I get dropped into an emergency shell). But it gets beyond the PXE boot and not being able to find a bootable volume. Anyhow, maybe someone has a pointer/idea what to look for?

[afbjorklund]

If the base disk is a LiveCD, like in the alpine.yaml, then the diff disk is a normal disk image.

basedisk: DOS/MBR boot sector; partition 2 : ID=0xef, start-CHS (0x3ff,254,63), end-CHS (0x3ff,254,63), startsector 232, 2880 sectors
diffdisk: QEMU QCOW2 Image (v3), 107374182400 bytes

lima-alpine:/home/anders/lima$ sudo findmnt /dev/vda1
TARGET     SOURCE                                     FSTYPE OPTIONS
/mnt/data  /dev/disk/by-label/data-volume             ext4   rw,relatime
/etc       /dev/disk/by-label/data-volume[/etc]       ext4   rw,relatime
/home      /dev/disk/by-label/data-volume[/home]      ext4   rw,relatime
/root      /dev/disk/by-label/data-volume[/root]      ext4   rw,relatime
/tmp       /dev/disk/by-label/data-volume[/tmp]       ext4   rw,relatime
/usr/local /dev/disk/by-label/data-volume[/usr/local] ext4   rw,relatime
/var/lib   /dev/disk/by-label/data-volume[/var/lib]   ext4   rw,relatime
lima-alpine:/home/anders/lima$ sudo df -h /mnt/data
Filesystem                Size      Used Available Use% Mounted on
/dev/disk/by-label/data-volume
                         97.9G      9.0M     92.8G   0% /mnt/data

If the base disk is a normal disk image, like in ubuntu.yaml, then the diff disk is a COW image.

basedisk: QEMU QCOW2 Image (v2), 2361393152 bytes
diffdisk: QEMU QCOW2 Image (v3), has backing file (path /home/anders/.lima/default/basedisk), 107374182400 bytes

anders@lima-default:/home/anders/lima$ sudo findmnt /dev/vda1
TARGET SOURCE    FSTYPE OPTIONS
/      /dev/vda1 ext4   rw,relatime,discard,errors=remount-ro
anders@lima-default:/home/anders/lima$ sudo df -h /
Filesystem      Size  Used Avail Use% Mounted on
/dev/vda1        97G  2.1G   95G   3% /

---

The boot format is UEFI by default, so if you only support BIOS then you need to set the config:

firmware:
  legacyBIOS: true

There seems to be a "flatcar_production_qemu_uefi_image.img", so maybe that would work better ?

[afbjorklund]

Do you mean something like the kernel.cmdline, for the kernel option ?

There seems to be some custom firmware, maybe that is required as well

VM_PFLASH_RO='flatcar_production_qemu_uefi_efi_code.fd'
VM_PFLASH_RW='flatcar_production_qemu_uefi_efi_vars.fd'

[till]

I have to understand/learn if that would do, haven't done that before. 🙈 I'll report back.

[afbjorklund]

It seemed to boot OK here, though. At least up until Booting 'Flatcar default' or so, didn't bother with any configuration...

qemu-system-x86_64 -bios /usr/share/OVMF/OVMF_CODE.fd -m 512 -hda flatcar_production_qemu_uefi_image.img

[till]

@afbjorklund most of this works without lima, but e.g. I can't find it using -boot in the code. What do you suggest?

[afbjorklund]

Maybe it needs more code ? But what was needed, order