Skip to content
/ run_qemu Public

A script to create bootable OS images, and run qemu with a locally built kernel.

License

Notifications You must be signed in to change notification settings

pmem/run_qemu

Repository files navigation

Requirements

  • mkosi
    • e.g. dnf install mkosi
  • qemu-system-x86_64
  • rsync
  • dracut
  • enabled virtualization (KVM)
  • nopasswd sudo preferred, or run as root, or enter passwords into the prompt several times
  • argbash to generate the argument parser lib (using parser_generator.m4)

Installation

  • symlink the run_qemu.sh script into somewhere in your PATH
    • e.g. ln -s ~/git/run_qemu/run_qemu.sh ~/bin/run_qemu.sh
  • Note: Supporting files in this repo are required to be in the same location as the script, after any symlink resolution. Copying just the script itself will not work.
  • Bash Completion
    • Copy or symlink the run_qemu file into the default bash completions dir
    • The completions directory can be found using: pkg-config --variable=completionsdir bash-completion

Usage Notes

  • Run this from the top level of a Linux kernel tree

    • e.g. run_qemu.sh --cxl --git-qemu
  • The script can/will:

    • Build the kernel with whatever .config is present (It is up to the user to manage the .config)
    • Create a rootfs image with the chosen distro using mkosi
    • Perform some basic setup on the rootfs, including installing the kernel, utilities (such as ndctl), and other convenience operations such as copying your ~/.ssh/id_rsa.pub for easy ssh access, and your ~/.bashrc etc.
    • Boot qemu with the newly compiled kernel provided on the qemu command line, and using the rootfs image above
    • Various options influence the qemu command line generated - there are options to select NUMA config, NVDIMMs, NVME devices, CXL devices etc.
  • More detailed CLI help is available with run_qemu.sh --help

  • Once qemu starts, in nographic mode, the Linux console 'takes over' the terminal. To interact with it, the following are useful:

    • Ctrl-a c : switch between the qemu monitor prompt (qemu) and console
    • Ctrl-a x : kill qemu and exit
  • mkosi creates a package cache in mkosi.cache/ If a cache is present, it will always use only that, and never go over the network even if newer packages are available. To force re-fetching everything, remove this directory, or --rebuild=wipe which removes the builddir entirely.

  • Which qemu to use can be overridden from the environment: qemu=/path/to/qemu/build/qemu-system-x86_64 ./run_qemu.sh [options]

  • List of variables that have overrides via env:

    • qemu
    • gdb
    • distro
    • rev
    • builddir
    • ndctl
  • To use the 'hostfwd' network, put this in your .ssh/config:

    Host rq
    Hostname localhost
    User root
    Port 10022
    StrictHostKeyChecking no
    UserKnownHostsFile /dev/null
    

    And then ssh rq. You may need to open port 10022 on any local firewalls.

  • The root password for the guest VM is root. The serial console automatically logs in, and a password isn't required.

CXL Usage

The script enables generating a sane QEMU commandline for instantiating a basic CXL topology. Since QEMU support for CXL isn't yet upstream, --git-qemu is additionally required. The CXL related options are:

  • --cxl: Enables a simple CXL topology with:
    • single host bridge
      • 512M window size at 0x4c00000000
      • Bus #52
    • single root port
    • single Type 3 device
      • Persistent 256M
    • simple label storage area
  • --cxl-debug: Add any and all flags for extra debug (kernel and QEMU)
  • --cxl-hb: Turn q35 into a CXL capable Host bridge. Don't use this option unless you're working on support for this.
  • --cxl-test-run: Attempt to do a sanity test of the kernel and QEMU configuration.

DAX Usage

  • --dax-debug: Add any and all flags for extra debug of dax modules (kernel)

Kernel config

  • Make sure to Turn on CXL related options in the kernel's .config:
$ grep -i cxl .config
CONFIG_CXL_BUS=y
CONFIG_CXL_PCI=m
CONFIG_CXL_MEM_RAW_COMMANDS=y
CONFIG_CXL_ACPI=m
CONFIG_CXL_PMEM=m
CONFIG_CXL_MEM=m
CONFIG_CXL_PORT=y
CONFIG_CXL_SUSPEND=y

The following is a way to check basic sanity within the QEMU guest:

lspci  | grep '3[45]:00'
34:00.0 PCI bridge: Intel Corporation Device 7075
35:00.0 Memory controller [0502]: Intel Corporation Device 0d93 (rev 01)

readlink -f /sys/bus/cxl/devices/mem0
/sys/devices/pci0000:34/0000:34:00.0/0000:35:00.0/mem0

Fedora 39+ Notes

Fedora 39 updates mkosi to 15 (and later) which contains a lot of breaking changes, and indeed breaks various expectations with run_qemu's usage of it.

Fedora has packaged mkosi 14 separately and in parallel to the latest mkosi. Uninstall the default mkosi package, and install 'mkosi14' to continue using run_qemu on Fedora 39 and later.

# dnf remove mkosi
# dnf install mkosi14

About

A script to create bootable OS images, and run qemu with a locally built kernel.

Topics

Resources

License

Security policy

Stars

Watchers

Forks

Languages