[WIP]: Support for libkrun using krunkit
#4137
Draft
+344
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nirs
reviewed
Oct 4, 2025
pkg/driver/krun/krun_darwin.go
Outdated
| "--cpus", fmt.Sprintf("%d", *inst.Config.CPUs), | ||
| "--device", fmt.Sprintf("virtio-serial,logFilePath=%s", filepath.Join(inst.Dir, filenames.SerialLog)), | ||
| "--krun-log-level", logLevelInfo, | ||
| "--restful-uri", fmt.Sprintf("unix://%s", restfulSocketPath(inst)), |
There was a problem hiding this comment.
The restful service is not very useful in krunkit:
- It always return "running"
- It may fail to respond if krunkit is terminating
- You cannot depend on it for stopping the vm since the vm may ignore the ACPI request
- Sending Stop request send an async event to libkrun and return immediately without waiting for ack from the library
For these reasons in minikube
- we use
--restfu-uri none://(which is also the default in latest krunkit, which is required to for using --device virtio-net,type=unixstream,path=...). - the vm state is the process state if the process exist the vm is running, if the process does not exist the vm is stopped
- we terminate the vm with SIGTERM
When the service will improved, it can make sense to use it for stopping the service gracefully, and terminate krunkit after a timeout if it failed to stop within a timeout.
Checking the status will always be the existence of the process. If the process exist you cannot start another instance, and if you want to stop it you must terminate the process anyway.
pkg/driver/krun/krun_darwin.go
Outdated
| "--device", fmt.Sprintf("virtio-serial,logFilePath=%s", filepath.Join(inst.Dir, filenames.SerialLog)), | ||
| "--krun-log-level", logLevelInfo, | ||
| "--restful-uri", fmt.Sprintf("unix://%s", restfulSocketPath(inst)), | ||
| "--bootloader", fmt.Sprintf("efi,variable-store=%s,create", filepath.Join(inst.Dir, KrunEfi)), |
There was a problem hiding this comment.
This is not needed - krunkit ignore this parameter. It exists for compatibility with vfkit.
| "--krun-log-level", logLevelInfo, | ||
| "--restful-uri", fmt.Sprintf("unix://%s", restfulSocketPath(inst)), | ||
| "--bootloader", fmt.Sprintf("efi,variable-store=%s,create", filepath.Join(inst.Dir, KrunEfi)), | ||
| "--device", fmt.Sprintf("virtio-blk,path=%s,format=raw", filepath.Join(inst.Dir, filenames.DiffDisk)), |
There was a problem hiding this comment.
Worth a comment that this is the boot disk. The order of the device matters.
pkg/driver/krun/krun_darwin.go
Outdated
| if err != nil { | ||
| return nil, err | ||
| } | ||
| networkArg := fmt.Sprintf("virtio-net,type=unixstream,path=%s,mac=%s,offloading=true", |
There was a problem hiding this comment.
offloading=true cannot not work with socket_vment since it does not enable offloading in the vment interface, and we really don't want to enable offloading in vment since it lead to horrible performance.
See https://github.com/containers/krunkit/blob/main/docs/usage.md#offloading-performance-implications
Remove the offloating=true parameter to make the network work. I tested it with socket_vment here:
containers/krunkit#63 (comment)
| } | ||
| if err = diskUtil.ConvertToRaw(ctx, baseDisk, diffDisk, &diskSize, false); err != nil { | ||
| return fmt.Errorf("failed to convert %q to a raw disk %q: %w", baseDisk, diffDisk, err) | ||
| } |
There was a problem hiding this comment.
krunkit support qcow2 disks but using raw disk is likely to perform better. We can add a comment here.
pkg/driver/krun/krun_darwin.go
Outdated
| // SPDX-FileCopyrightText: Copyright The Lima Authors | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| package krun |
There was a problem hiding this comment.
I'm not sure about the name. The program we use here is named krunkit, and the library is called libkrun. In minikube the driver is called krunkit similar to other drivers (vfkit, qemu). In lima we call the qemu driver qemu, so this driver should call krunkit for consistency.
There was a problem hiding this comment.
The "Kit" suffix is carried over from Docker, where it was HyperKit and LinuxKit and BuildKit and so on
| Instance *limatype.Instance | ||
| SSHLocalPort int | ||
|
|
||
| krunCmd *exec.Cmd |
There was a problem hiding this comment.
krunCmd is not a good name since the command is krunkit. This is internal detail not visible to users, but it makes the code more confusing.
| "github.com/lima-vm/lima/v2/pkg/ptr" | ||
| ) | ||
|
|
||
| type LimaKrunDriver struct { |
There was a problem hiding this comment.
Why do we need the Lima prefix? Do we use LimaQemuDriver for the qemu driver?
There was a problem hiding this comment.
Do we use LimaQemuDriver for the qemu driver?
Yes
| return nil | ||
| } | ||
|
|
||
| if err := l.krunCmd.Process.Signal(os.Interrupt); err != nil { |
There was a problem hiding this comment.
Use SIGTERM for terminating a process
| return nil | ||
| case <-timeout: | ||
| return l.krunCmd.Process.Kill() | ||
| } |
There was a problem hiding this comment.
You need to wait for the process after killing it. Otherwise it will remain a zombie until this process terminates.
afbjorklund
reviewed
Oct 5, 2025
pkg/driver/krun/register.go
Outdated
| import "github.com/lima-vm/lima/v2/pkg/registry" | ||
|
|
||
| func init() { | ||
| registry.Register(New()) |
There was a problem hiding this comment.
This is only needed for built-in internal drivers, it is not needed for external drivers.
Signed-off-by: Ansuman Sahoo <[email protected]>
|
@lima-vm/maintainers @lima-vm/committers Any help is appreciated! What am I doing wrong here that an IP is not getting assigned for the For # limactl start default --vm-type=qemu --network=lima:shared
➜ lima2 git:(driver/krun) ✗ arp -an | grep bridge100
? (192.168.105.1) at 52:ed:3c:d4:1e:64 on bridge100 ifscope permanent [bridge]
? (192.168.105.2) at 52:55:55:e0:48:67 on bridge100 ifscope [bridge] # VM IP
? (224.0.0.251) at 1:0:5e:0:0:fb on bridge100 ifscope permanent [ethernet]For ➜ limactl start default --vm-type=krunkit --network=lima:shared
WARN[0000] Template locator "template://_images/ubuntu" should be written "template:_images/ubuntu" since Lima v2.0
WARN[0000] Template locator "template://_default/mounts" should be written "template:_default/mounts" since Lima v2.0
? Creating an instance "default" Proceed with the current configuration
INFO[0001] Starting socket_vmnet daemon for "shared" network
INFO[0001] Starting the instance "default" with external VM driver "krunkit"
INFO[0001] Attempting to download the image arch=aarch64 digest="sha256:26d0ac2236f12954923eb35ddfee8fa9fff3eab6111ba84786b98ab3b972c6d8" location="https://cloud-images.ubuntu.com/releases/plucky/release-20250701/ubuntu-25.04-server-cloudimg-arm64.img"
INFO[0001] Using cache "/Users/ansumansahoo/Library/Caches/lima/download/by-url-sha256/eccac025a7a4709721a97d4ef1d5973546c85062917f188cdc6d247b665cbab5/data"
INFO[0004] Attempting to download the nerdctl archive arch=aarch64 digest="sha256:2f98346ed3dcaf47cfd6461a5685e582284329a1ece1d952ade881b9fe7491e8" location="https://github.com/containerd/nerdctl/releases/download/v2.1.6/nerdctl-full-2.1.6-linux-arm64.tar.gz"
INFO[0004] Using cache "/Users/ansumansahoo/Library/Caches/lima/download/by-url-sha256/d82077f1308e8d063805c4c1f761e1c3fd636e18cab7e25919a18e7a0838afa2/data"
INFO[0006] [hostagent] hostagent socket created at /Users/ansumansahoo/.lima/default/ha.sock
INFO[0006] SSH Local Port: 55283
INFO[0006] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
...
INFO[0598] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0598] [hostagent] Waiting for the essential requirement 2 of 2: "user session is ready for ssh"
FATA[0604] did not receive an event with the "running" status
➜ lima2 git:(driver/krun) ✗ arp -an | grep bridge100
? (192.168.105.1) at 52:ed:3c:d4:1e:64 on bridge100 ifscope permanent [bridge]
? (224.0.0.251) at 1:0:5e:0:0:fb on bridge100 ifscope permanent [ethernet]Also tried this but didn't work: https://github.com/lima-vm/socket_vmnet?tab=readme-ov-file#ip-address-is-not-assigned Is it because the Logs for
|
nirs
reviewed
Oct 13, 2025
| } | ||
| networkArg := fmt.Sprintf("virtio-net,type=unixstream,path=%s,mac=%s", | ||
| sock, | ||
| limayaml.MACAddress(inst.Dir), |
There was a problem hiding this comment.
This looks correct now, so krunkit should be connected via socket_vment with this change and works similar to qemu.
| return nil, err | ||
| } | ||
| if socketVMNetOk { | ||
| sock, err := networks.Sock(networks.ModeShared) |
There was a problem hiding this comment.
Why hard code shared mode? this driver should be able to use both shared and bridged modes. It is fined to start with shared mode initially, but the final driver should support both shared and bridged. Since we really require socket_vment, we can fail if the instance is not using vment.
| return args, nil | ||
| } | ||
|
|
||
| return args, errors.New("socket_vmnet is not installed") |
There was a problem hiding this comment.
Returning the error at the end is not good. The common pattern we should always follow is the happy path:
if condition not met:
return nil, err
if another condition to met:
return nil, err
...
return args, nil
This makes error handling much easier to understand since the error message is close to the check triggering it. It also help to be sure we handled all errors, and it keeps indentation simple and flat, more important in go, forcing tabs (8 spaces) for every indentation level.
| if err != nil { | ||
| return nil, err | ||
| } | ||
| socketVMNetOk, err := nwCfg.IsDaemonInstalled(networks.SocketVMNet) |
There was a problem hiding this comment.
Let's rename the argument to make the code more clear:
socketVMNetInstalled, err := nwCfg.IsDaemonInstalled(networks.SocketVMNet)
| if err != nil { | ||
| return nil, err | ||
| } | ||
| if socketVMNetOk { |
There was a problem hiding this comment.
Lets change this to:
if ! socketVMNetInstalled {
return args, errors.New("socket_vmnet is not installed")
}
| if err != nil { | ||
| return nil, fmt.Errorf("failed to construct krunkit command line: %w", err) | ||
| } | ||
| krunCmd.SysProcAttr = executil.BackgroundSysProcAttr |
There was a problem hiding this comment.
Commenting why we need this will be helpful, you can use the same comment used for other drivers.
| return &LimaKrunDriver{} | ||
| } | ||
|
|
||
| func (l *LimaKrunDriver) Configure(inst *limatype.Instance) *driver.ConfiguredDriver { |
There was a problem hiding this comment.
Do we use l in other drivers? this is not great since it look like 1 and should be avoided in general. But it is also not great to use the l (for Lima) in this context, since this is a Driver function, so the right short name is d.
If we use this in other drivers we should really change to d in all drivers. Since this is a new driver it can be ok to use d in this driver.
There was a problem hiding this comment.
Do we use
lin other drivers? If we use this in other drivers we should really change to d in all drivers.
Yes it can be another issue but can be done for krunkit!
arp is not a good way to find the ip, it can take a while until a linux vm is available via arp.
This is not needed since macOS 15, I guess you are on macOS 15 or 26?
The guest agent should not affect this. The vm should get an ip address in few seconds after it starts. Typical when it does not work it is because an issue in vmnet/socketlifterfw,bootpd/kernel. Unfortunately we have very little visibility on the internals.
Good, we know that the vm started.
Looks good
Looks good if I remember correctly. Stuff you can try when you don't get ip address:
If nothing else works, usually reboot fixes the issue. |
|
I tested with socket_vment installed as launchd daemon - I prefer to use a proper daemon instead of manage the daemon using lima with sudo. Using this yaml: And nothing happens - looks like logging errors on startup is broken. I did not try to look at logs since I'm not interested in "lima: shared" mode. After changing the network to: lima starts: And krunkit is running as: This cannot work since lima does not create the socket: And krunkit panics: We need fix krunkit the check if the path exist and fail in a cleaner way. Checking the qemu driver network code I found that we use the wrong Mac address for the device - the Mac address of the user network instead of the socket_vmnet network. I fixed the networking arguments using: diff --git a/pkg/driver/krunkit/krunkit_darwin.go b/pkg/driver/krunkit/krunkit_darwin.go
index db930539..c200f4be 100644
--- a/pkg/driver/krunkit/krunkit_darwin.go
+++ b/pkg/driver/krunkit/krunkit_darwin.go
@@ -17,7 +17,6 @@ import (
"github.com/lima-vm/lima/v2/pkg/iso9660util"
"github.com/lima-vm/lima/v2/pkg/limatype"
"github.com/lima-vm/lima/v2/pkg/limatype/filenames"
- "github.com/lima-vm/lima/v2/pkg/limayaml"
"github.com/lima-vm/lima/v2/pkg/networks"
)
@@ -52,6 +51,8 @@ func Cmdline(inst *limatype.Instance) (*exec.Cmd, error) {
func buildNetworkArgs(inst *limatype.Instance) ([]string, error) {
var args []string
+
+ // We support only socket_vment - check if we can use it.
nwCfg, err := networks.LoadConfig()
if err != nil {
return nil, err
@@ -60,21 +61,32 @@ func buildNetworkArgs(inst *limatype.Instance) ([]string, error) {
if err != nil {
return nil, err
}
- if socketVMNetOk {
- sock, err := networks.Sock(networks.ModeShared)
- if err != nil {
- return nil, err
+ if !socketVMNetOk {
+ return args, errors.New("socket_vmnet is not installed")
+ }
+
+ // Add one or more network devices.
+ for _, nw := range inst.Networks {
+ var sock string
+ if nw.Lima == networks.ModeShared || nw.Lima == networks.ModeBridged {
+ sock, err = networks.Sock(nw.Lima)
+ if err != nil {
+ return nil, err
+ }
+ } else if nw.Socket != "" {
+ sock = nw.Socket
+ } else {
+ return nil, fmt.Errorf("invalid network spec %+v", nw)
}
- networkArg := fmt.Sprintf("virtio-net,type=unixstream,path=%s,mac=%s",
- sock,
- limayaml.MACAddress(inst.Dir),
- )
- args = append(args, "--device", networkArg)
+ device := fmt.Sprintf("virtio-net,type=unixstream,path=%s,mac=%s", sock, nw.MACAddress)
+ args = append(args, "--device", device)
+ }
- return args, nil
+ if len(args) == 0 {
+ return args, fmt.Errorf("no socket_vment networks defined")
}
- return args, errors.New("socket_vmnet is not installed")
+ return args, nil
}
func EnsureDisk(ctx context.Context, inst *limatype.Instance) error {With this krunkit is started with the right arguments: But it takes about a minute until serial.log shows the login prompt, and krunkit does not get an ip address. The issue is that we generate this network_config, which cannot work: krunkit does not have a user network and we create only one virtio-net device. I'm not sure why this configuration does not work, since we don't have any device matching I found that I can remove the user network with this change: diff --git a/pkg/cidata/cidata.go b/pkg/cidata/cidata.go
index 888e17c5..dd0bdd0d 100644
--- a/pkg/cidata/cidata.go
+++ b/pkg/cidata/cidata.go
@@ -257,7 +257,11 @@ func templateArgs(ctx context.Context, bootScripts bool, instDir, name string, i
})
}
- args.Networks = append(args.Networks, Network{MACAddress: limayaml.MACAddress(instDir), Interface: networks.SlirpNICName, Metric: 200})
+ // XXX Should query the driver instead of hardcodeing the name.
+ if *instConfig.VMType != "krunkit" {
+ args.Networks = append(args.Networks, Network{MACAddress: limayaml.MACAddress(instDir), Interface: networks.SlirpNICName, Metric: 200})
+ }
+
for i, nw := range instConfig.Networks {
if i == firstUsernetIndex {
continueWith this change krunkit starts in few seconds and we can login: % ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=error -l nir 192.168.105.2 -i ~/.lima/_config/user
Welcome to Ubuntu 25.04 (GNU/Linux 6.14.0-23-generic aarch64)
...
nir@lima-krunkit:~$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host noprefixroute
valid_lft forever preferred_lft forever
2: lima0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 52:55:55:eb:bf:61 brd ff:ff:ff:ff:ff:ff
altname enx525555ebbf61
inet 192.168.105.2/24 metric 100 brd 192.168.105.255 scope global dynamic lima0
valid_lft 334sec preferred_lft 334sec
inet6 fe80::5055:55ff:feeb:bf61/64 scope link proto kernel_ll
valid_lft forever preferred_lft foreverBut limactl is stuck trying to access ssh via the user network which krunkit does not have: To integrate with krunkit we need to change lima to support driver without a user network:
|
Why can't we have a user network ? |
This comment was marked as off-topic.
This comment was marked as off-topic.
Actually we can! But the result is:
So having an option to run an instance without a user network is better. PatchWith this patch on top of this PR I could start in plain mode: diff --git a/pkg/driver/krunkit/krunkit_darwin.go b/pkg/driver/krunkit/krunkit_darwin.go
index db930539..d86a0c39 100644
--- a/pkg/driver/krunkit/krunkit_darwin.go
+++ b/pkg/driver/krunkit/krunkit_darwin.go
@@ -19,6 +19,7 @@ import (
"github.com/lima-vm/lima/v2/pkg/limatype/filenames"
"github.com/lima-vm/lima/v2/pkg/limayaml"
"github.com/lima-vm/lima/v2/pkg/networks"
+ "github.com/lima-vm/lima/v2/pkg/networks/usernet"
)
const (
@@ -52,29 +53,55 @@ func Cmdline(inst *limatype.Instance) (*exec.Cmd, error) {
func buildNetworkArgs(inst *limatype.Instance) ([]string, error) {
var args []string
- nwCfg, err := networks.LoadConfig()
- if err != nil {
- return nil, err
- }
- socketVMNetOk, err := nwCfg.IsDaemonInstalled(networks.SocketVMNet)
- if err != nil {
- return nil, err
- }
- if socketVMNetOk {
- sock, err := networks.Sock(networks.ModeShared)
- if err != nil {
- return nil, err
+
+ // Add one or more network devices.
+ for _, nw := range inst.Networks {
+ var sock string
+ var mac string
+
+ if nw.Lima != "" {
+ nwCfg, err := networks.LoadConfig()
+ if err != nil {
+ return nil, err
+ }
+ switch nw.Lima {
+ case networks.ModeUserV2:
+ sock, err = usernet.Sock(nw.Lima, usernet.QEMUSock)
+ if err != nil {
+ return nil, err
+ }
+ mac = limayaml.MACAddress(inst.Dir)
+ case networks.ModeShared, networks.ModeBridged:
+ socketVMNetOk, err := nwCfg.IsDaemonInstalled(networks.SocketVMNet)
+ if err != nil {
+ return nil, err
+ }
+ if !socketVMNetOk {
+ return args, errors.New("socket_vmnet is not installed")
+ }
+ sock, err = networks.Sock(nw.Lima)
+ if err != nil {
+ return nil, err
+ }
+ mac = nw.MACAddress
+ default:
+ return nil, fmt.Errorf("invalid network spec %+v", nw)
+ }
+ } else if nw.Socket != "" {
+ sock = nw.Socket
+ mac = nw.MACAddress
+ } else {
+ return nil, fmt.Errorf("invalid network spec %+v", nw)
}
- networkArg := fmt.Sprintf("virtio-net,type=unixstream,path=%s,mac=%s",
- sock,
- limayaml.MACAddress(inst.Dir),
- )
- args = append(args, "--device", networkArg)
+ device := fmt.Sprintf("virtio-net,type=unixstream,path=%s,mac=%s", sock, mac)
+ args = append(args, "--device", device)
+ }
- return args, nil
+ if len(args) == 0 {
+ return args, fmt.Errorf("no socket_vment networks defined")
}
- return args, errors.New("socket_vmnet is not installed")
+ return args, nil
}
func EnsureDisk(ctx context.Context, inst *limatype.Instance) error {
diff --git a/pkg/driver/krunkit/krunkit_driver_darwin.go b/pkg/driver/krunkit/krunkit_driver_darwin.go
index bf5d0a2c..a4bfa7c3 100644
--- a/pkg/driver/krunkit/krunkit_driver_darwin.go
+++ b/pkg/driver/krunkit/krunkit_driver_darwin.go
@@ -19,6 +19,8 @@ import (
"github.com/lima-vm/lima/v2/pkg/executil"
"github.com/lima-vm/lima/v2/pkg/limatype"
"github.com/lima-vm/lima/v2/pkg/limatype/filenames"
+ "github.com/lima-vm/lima/v2/pkg/limayaml"
+ "github.com/lima-vm/lima/v2/pkg/networks/usernet"
"github.com/lima-vm/lima/v2/pkg/ptr"
)
@@ -86,7 +88,15 @@ func (l *LimaKrunDriver) Start(ctx context.Context) (chan error, error) {
}()
l.krunkitWaitCh <- krunCmd.Wait()
}()
-
+ go func() {
+ if usernetIndex := limayaml.FirstUsernetIndex(l.Instance.Config); usernetIndex != -1 {
+ client := usernet.NewClientByName(l.Instance.Config.Networks[usernetIndex].Lima)
+ err := client.ConfigureDriver(ctx, l.Instance, l.SSHLocalPort)
+ if err != nil {
+ l.krunkitWaitCh <- err
+ }
+ }
+ }()
return l.krunkitWaitCh, nil
}
ConfigUsing this config: vmType: krunkit
images:
- location: "https://cloud-images.ubuntu.com/releases/plucky/release-20250701/ubuntu-25.04-server-cloudimg-arm64.img"
arch: "aarch64"
digest: "sha256:26d0ac2236f12954923eb35ddfee8fa9fff3eab6111ba84786b98ab3b972c6d8"
- location: https://cloud-images.ubuntu.com/releases/plucky/release/ubuntu-25.04-server-cloudimg-arm64.img
arch: aarch64
plain: true
portForwards:
- static: true
guestPort: 5201
proto: tcp
networks:
# Required for boot
- lima: user-v2
# Make it accesisble from host and other instances
- socket: /var/run/socket_vmnet
metric: 300 # prefer user-v2lima: user-v2 is required for the ssh connection with current lima. We must change the metric on the socket network to make user-v2 preferred. Setting metric on the lima network does not work. Without I don't have network access from the instance. Testing krunkit driverStarting: % ./_output/bin/limactl start krunkit
INFO[0000] Using the existing instance "krunkit"
INFO[0000] Starting the instance "krunkit" with external VM driver "krunkit"
INFO[0000] [hostagent] hostagent socket created at /Users/nir/.lima/krunkit/ha.sock
INFO[0000] SSH Local Port: 51680
INFO[0000] [hostagent] Running in plain mode. Mounts, dynamic port forwarding, containerd, etc. will be ignored. Guest agent will not be running. Static port forwarding is allowed.
INFO[0000] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0010] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0010] [hostagent] The essential requirement 1 of 2 is satisfied
INFO[0010] [hostagent] Waiting for the essential requirement 2 of 2: "Explicitly start ssh ControlMaster"
INFO[0010] [hostagent] The essential requirement 2 of 2 is satisfied
INFO[0010] [hostagent] Setting up static TCP forwarding from 127.0.0.1:5201 to 127.0.0.1:5201
INFO[0011] [hostagent] Waiting for the final requirement 1 of 1: "boot scripts must have finished"
INFO[0011] [hostagent] The final requirement 1 of 1 is satisfied
INFO[0011] READY. Run `ssh -F "/Users/nir/.lima/krunkit/ssh.config" lima-krunkit` to open the shell. Lima processes: % ps | grep lima
21492 ttys005 0:00.01 grep lima
21334 ttys006 0:20.70 /Users/nir/src/lima/_output/bin/limactl usernet -p /Users/nir/.lima/_networks/user-v2/usernet_user-v2.pid -e /Users/nir/.lima/_networks/user-v2/user-v2_ep.sock --listen-qemu /Users/nir/.lima/_networks/user-v2/user-v2_qemu.sock --listen /Users/nir/.lima/_networks/user-v2/user-v2_fd.sock --subnet 192.168.104.0/24 --leases 192.168.104.1=52:55:55:f3:38:6a,192.168.104.2=5a:94:ef:e4:0c:dd,192.168.104.3=52:55:55:7d:08:f5,192.168.104.4=52:55:55:92:cc:15
21336 ttys006 0:00.08 /Users/nir/src/lima/_output/bin/limactl hostagent --pidfile /Users/nir/.lima/krunkit/ha.pid --socket /Users/nir/.lima/krunkit/ha.sock krunkit
21340 ttys006 0:00.03 /Users/nir/src/lima/_output/libexec/lima/lima-driver-krunkit
21345 ttys006 0:29.40 krunkit --memory 2048 --cpus 4 --device virtio-serial,logFilePath=/Users/nir/.lima/krunkit/serial.log --krun-log-level 3 --restful-uri none:// --device virtio-blk,path=/Users/nir/.lima/krunkit/diffdisk,format=raw --device virtio-blk,path=/Users/nir/.lima/krunkit/cidata.iso --device virtio-net,type=unixstream,path=/Users/nir/.lima/_networks/user-v2/user-v2_qemu.sock,mac=52:55:55:92:cc:15 --device virtio-net,type=unixstream,path=/var/run/socket_vmnet,mac=52:55:55:12:88:1fAccessing via limactl shell % ./_output/bin/limactl shell krunkit
nir@lima-krunkit:~$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host noprefixroute
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 52:55:55:92:cc:15 brd ff:ff:ff:ff:ff:ff
altname enx52555592cc15
inet 192.168.104.4/24 metric 200 brd 192.168.104.255 scope global dynamic eth0
valid_lft 3512sec preferred_lft 3512sec
inet6 fe80::5055:55ff:fe92:cc15/64 scope link proto kernel_ll
valid_lft forever preferred_lft forever
3: lima1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 52:55:55:12:88:1f brd ff:ff:ff:ff:ff:ff
altname enx52555512881f
inet 192.168.105.3/24 metric 300 brd 192.168.105.255 scope global dynamic lima1
valid_lft 512sec preferred_lft 512sec
inet6 fe80::5055:55ff:fe12:881f/64 scope link proto kernel_ll
valid_lft forever preferred_lft forever
nir@lima-krunkit:~$ ip r
default via 192.168.104.2 dev eth0 proto dhcp src 192.168.104.4 metric 200
default via 192.168.105.1 dev lima1 proto dhcp src 192.168.105.3 metric 300
192.168.104.0/24 dev eth0 proto kernel scope link src 192.168.104.4 metric 200
192.168.104.2 dev eth0 proto dhcp scope link src 192.168.104.4 metric 200
192.168.105.0/24 dev lima1 proto kernel scope link src 192.168.105.3 metric 300
192.168.105.1 dev lima1 proto dhcp scope link src 192.168.105.3 metric 300 Testing socket_vmnet: % iperf3 -c 192.168.105.3
Connecting to host 192.168.105.3, port 5201
[ 5] local 192.168.105.1 port 51554 connected to 192.168.105.3 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.01 sec 479 MBytes 4.00 Gbits/sec
[ 5] 1.01-2.01 sec 466 MBytes 3.91 Gbits/sec
[ 5] 2.01-3.00 sec 474 MBytes 3.99 Gbits/sec
[ 5] 3.00-4.01 sec 481 MBytes 4.02 Gbits/sec
[ 5] 4.01-5.01 sec 483 MBytes 4.05 Gbits/sec
[ 5] 5.01-6.01 sec 484 MBytes 4.06 Gbits/sec
[ 5] 6.01-7.01 sec 470 MBytes 3.94 Gbits/sec
[ 5] 7.01-8.01 sec 470 MBytes 3.95 Gbits/sec
[ 5] 8.01-9.01 sec 464 MBytes 3.89 Gbits/sec
[ 5] 9.01-10.01 sec 478 MBytes 4.01 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.01 sec 4.64 GBytes 3.98 Gbits/sec sender
[ 5] 0.00-10.01 sec 4.64 GBytes 3.98 Gbits/sec receiverTesting user-v2: % iperf3 -c localhost
Connecting to host localhost, port 5201
[ 7] local 127.0.0.1 port 51691 connected to 127.0.0.1 port 5201
[ ID] Interval Transfer Bitrate
[ 7] 0.00-1.00 sec 266 MBytes 2.23 Gbits/sec
[ 7] 1.00-2.00 sec 260 MBytes 2.18 Gbits/sec
[ 7] 2.00-3.00 sec 259 MBytes 2.17 Gbits/sec
[ 7] 3.00-4.00 sec 260 MBytes 2.18 Gbits/sec
[ 7] 4.00-5.00 sec 264 MBytes 2.21 Gbits/sec
[ 7] 5.00-6.00 sec 280 MBytes 2.34 Gbits/sec
[ 7] 6.00-7.00 sec 259 MBytes 2.18 Gbits/sec
[ 7] 7.00-8.00 sec 260 MBytes 2.18 Gbits/sec
[ 7] 8.00-9.00 sec 263 MBytes 2.21 Gbits/sec
[ 7] 9.00-10.00 sec 259 MBytes 2.17 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 7] 0.00-10.00 sec 2.57 GBytes 2.20 Gbits/sec sender
[ 7] 0.00-10.01 sec 2.56 GBytes 2.20 Gbits/sec receiverTesting multiple instancesI created another instance with same configuration without static port forwarding: vmType: krunkit
images:
- location: "https://cloud-images.ubuntu.com/releases/plucky/release-20250701/ubuntu-25.04-server-cloudimg-arm64.img"
arch: "aarch64"
digest: "sha256:26d0ac2236f12954923eb35ddfee8fa9fff3eab6111ba84786b98ab3b972c6d8"
- location: https://cloud-images.ubuntu.com/releases/plucky/release/ubuntu-25.04-server-cloudimg-arm64.img
arch: aarch64
plain: true
networks:
- lima: user-v2
- socket: /var/run/socket_vmnet
metric: 300Running iperf3 on second instance (krunkit2) connecting to first (krunkit) via the instance IP (socket_vment) or the internal DNS name on the user-v2 network. nir@lima-krunkit2:~$ iperf3 -c 192.168.105.3
Connecting to host 192.168.105.3, port 5201
[ 5] local 192.168.105.4 port 54540 connected to 192.168.105.3 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 297 MBytes 2.49 Gbits/sec 0 3.95 MBytes
[ 5] 1.00-2.00 sec 304 MBytes 2.55 Gbits/sec 0 3.95 MBytes
[ 5] 2.00-3.00 sec 308 MBytes 2.59 Gbits/sec 0 3.95 MBytes
[ 5] 3.00-4.00 sec 306 MBytes 2.57 Gbits/sec 0 3.95 MBytes
[ 5] 4.00-5.00 sec 307 MBytes 2.57 Gbits/sec 0 3.95 MBytes
[ 5] 5.00-6.00 sec 310 MBytes 2.60 Gbits/sec 0 3.95 MBytes
[ 5] 6.00-7.00 sec 307 MBytes 2.58 Gbits/sec 0 3.95 MBytes
[ 5] 7.00-8.00 sec 304 MBytes 2.55 Gbits/sec 0 3.95 MBytes
[ 5] 8.00-9.00 sec 309 MBytes 2.59 Gbits/sec 0 3.95 MBytes
[ 5] 9.00-10.00 sec 306 MBytes 2.57 Gbits/sec 0 3.95 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 2.99 GBytes 2.57 Gbits/sec 0 sender
[ 5] 0.00-10.01 sec 2.99 GBytes 2.56 Gbits/sec receiver
nir@lima-krunkit2:~$ iperf3 -c lima-krunkit.internal
Connecting to host lima-krunkit.internal, port 5201
[ 5] local 192.168.104.5 port 37324 connected to 192.168.104.4 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 314 MBytes 2.63 Gbits/sec 0 4.00 MBytes
[ 5] 1.00-2.00 sec 319 MBytes 2.67 Gbits/sec 0 4.00 MBytes
[ 5] 2.00-3.00 sec 316 MBytes 2.65 Gbits/sec 0 4.00 MBytes
[ 5] 3.00-4.00 sec 315 MBytes 2.64 Gbits/sec 0 4.00 MBytes
[ 5] 4.00-5.00 sec 317 MBytes 2.66 Gbits/sec 0 4.21 MBytes
[ 5] 5.00-6.00 sec 317 MBytes 2.66 Gbits/sec 0 4.21 MBytes
[ 5] 6.00-7.00 sec 317 MBytes 2.66 Gbits/sec 0 4.21 MBytes
[ 5] 7.00-8.00 sec 319 MBytes 2.68 Gbits/sec 0 4.21 MBytes
[ 5] 8.00-9.00 sec 316 MBytes 2.65 Gbits/sec 0 4.21 MBytes
[ 5] 9.00-10.00 sec 315 MBytes 2.64 Gbits/sec 0 4.21 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 3.09 GBytes 2.65 Gbits/sec 0 sender
[ 5] 0.00-10.01 sec 3.09 GBytes 2.65 Gbits/sec receiverComparing to vzI replaced vmType to vz and run the same tests: % ./_output/bin/limactl start krunkit
INFO[0000] Using the existing instance "krunkit"
INFO[0000] Starting the instance "krunkit" with internal VM driver "vz"
INFO[0000] [hostagent] hostagent socket created at /Users/nir/.lima/krunkit/ha.sock
INFO[0000] [hostagent] Starting VZ (hint: to watch the boot progress, see "/Users/nir/.lima/krunkit/serial*.log")
INFO[0000] [hostagent] [VZ] - vm state change: running
INFO[0006] [hostagent] Started vsock forwarder: 127.0.0.1:51751 -> vsock:22 on VM
INFO[0006] [hostagent] Detected SSH server is listening on the vsock port; changed 127.0.0.1:51751 to proxy for the vsock port
INFO[0007] SSH Local Port: 51751
INFO[0006] [hostagent] Running in plain mode. Mounts, dynamic port forwarding, containerd, etc. will be ignored. Guest agent will not be running. Static port forwarding is allowed.
INFO[0006] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0016] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0017] [hostagent] The essential requirement 1 of 2 is satisfied
INFO[0017] [hostagent] Waiting for the essential requirement 2 of 2: "Explicitly start ssh ControlMaster"
INFO[0017] [hostagent] The essential requirement 2 of 2 is satisfied
INFO[0017] [hostagent] Setting up static TCP forwarding from 127.0.0.1:5201 to 127.0.0.1:5201
INFO[0017] [hostagent] Waiting for the final requirement 1 of 1: "boot scripts must have finished"
INFO[0017] [hostagent] The final requirement 1 of 1 is satisfied
INFO[0017] READY. Run `ssh -F "/Users/nir/.lima/krunkit/ssh.config" lima-krunkit` to open the shell.
% iperf3 -c 192.168.105.3
Connecting to host 192.168.105.3, port 5201
[ 5] local 192.168.105.1 port 51758 connected to 192.168.105.3 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 281 MBytes 2.36 Gbits/sec
[ 5] 1.00-2.00 sec 285 MBytes 2.39 Gbits/sec
[ 5] 2.00-3.00 sec 283 MBytes 2.38 Gbits/sec
[ 5] 3.00-4.00 sec 282 MBytes 2.37 Gbits/sec
[ 5] 4.00-5.00 sec 284 MBytes 2.37 Gbits/sec
[ 5] 5.00-6.00 sec 282 MBytes 2.36 Gbits/sec
[ 5] 6.00-7.00 sec 282 MBytes 2.38 Gbits/sec
[ 5] 7.00-8.00 sec 283 MBytes 2.37 Gbits/sec
[ 5] 8.00-9.00 sec 282 MBytes 2.36 Gbits/sec
[ 5] 9.00-10.00 sec 282 MBytes 2.37 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.00 sec 2.76 GBytes 2.37 Gbits/sec sender
[ 5] 0.00-9.92 sec 2.76 GBytes 2.39 Gbits/sec receiver
% iperf3 -c localhost
Connecting to host localhost, port 5201
[ 7] local 127.0.0.1 port 51760 connected to 127.0.0.1 port 5201
[ ID] Interval Transfer Bitrate
[ 7] 0.00-1.00 sec 549 MBytes 4.61 Gbits/sec
[ 7] 1.00-2.00 sec 543 MBytes 4.55 Gbits/sec
[ 7] 2.00-3.00 sec 543 MBytes 4.56 Gbits/sec
[ 7] 3.00-4.00 sec 548 MBytes 4.59 Gbits/sec
[ 7] 4.00-5.00 sec 546 MBytes 4.58 Gbits/sec
[ 7] 5.00-6.00 sec 549 MBytes 4.60 Gbits/sec
[ 7] 6.00-7.00 sec 543 MBytes 4.56 Gbits/sec
[ 7] 7.00-8.00 sec 540 MBytes 4.53 Gbits/sec
[ 7] 8.00-9.00 sec 554 MBytes 4.65 Gbits/sec
[ 7] 9.00-10.00 sec 548 MBytes 4.60 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 7] 0.00-10.00 sec 5.33 GBytes 4.58 Gbits/sec sender
[ 7] 0.00-9.94 sec 5.33 GBytes 4.61 Gbits/sec receiverComparing to qemuI changed vmType to qemu and run the same tests. % ./_output/bin/limactl start krunkit
INFO[0000] Using the existing instance "krunkit"
INFO[0000] Starting the instance "krunkit" with internal VM driver "qemu"
INFO[0000] [hostagent] hostagent socket created at /Users/nir/.lima/krunkit/ha.sock
INFO[0000] [hostagent] Using system firmware ("/opt/homebrew/share/qemu/edk2-aarch64-code.fd")
INFO[0000] [hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/nir/.lima/krunkit/serial*.log")
INFO[0001] SSH Local Port: 51767
INFO[0000] [hostagent] Running in plain mode. Mounts, dynamic port forwarding, containerd, etc. will be ignored. Guest agent will not be running. Static port forwarding is allowed.
INFO[0000] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0010] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0010] [hostagent] The essential requirement 1 of 2 is satisfied
INFO[0010] [hostagent] Waiting for the essential requirement 2 of 2: "Explicitly start ssh ControlMaster"
INFO[0010] [hostagent] The essential requirement 2 of 2 is satisfied
INFO[0010] [hostagent] Setting up static TCP forwarding from 127.0.0.1:5201 to 127.0.0.1:5201
INFO[0010] [hostagent] Waiting for the final requirement 1 of 1: "boot scripts must have finished"
INFO[0010] [hostagent] The final requirement 1 of 1 is satisfied
INFO[0011] READY. Run `ssh -F "/Users/nir/.lima/krunkit/ssh.config" lima-krunkit` to open the shell.
% iperf3 -c 192.168.105.3
Connecting to host 192.168.105.3, port 5201
[ 5] local 192.168.105.1 port 51774 connected to 192.168.105.3 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 360 MBytes 3.02 Gbits/sec
[ 5] 1.00-2.00 sec 364 MBytes 3.06 Gbits/sec
[ 5] 2.00-3.00 sec 438 MBytes 3.67 Gbits/sec
[ 5] 3.00-4.00 sec 367 MBytes 3.08 Gbits/sec
[ 5] 4.00-5.00 sec 467 MBytes 3.92 Gbits/sec
[ 5] 5.00-6.01 sec 473 MBytes 3.95 Gbits/sec
[ 5] 6.01-7.00 sec 470 MBytes 3.96 Gbits/sec
[ 5] 7.00-8.01 sec 428 MBytes 3.57 Gbits/sec
[ 5] 8.01-9.00 sec 364 MBytes 3.07 Gbits/sec
[ 5] 9.00-10.01 sec 426 MBytes 3.56 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.01 sec 4.06 GBytes 3.49 Gbits/sec sender
[ 5] 0.00-10.08 sec 4.06 GBytes 3.46 Gbits/sec receiver
% iperf3 -c localhost
Connecting to host localhost, port 5201
[ 7] local 127.0.0.1 port 51777 connected to 127.0.0.1 port 5201
[ ID] Interval Transfer Bitrate
[ 7] 0.00-1.00 sec 227 MBytes 1.90 Gbits/sec
[ 7] 1.00-2.00 sec 223 MBytes 1.87 Gbits/sec
[ 7] 2.00-3.00 sec 272 MBytes 2.28 Gbits/sec
[ 7] 3.00-4.00 sec 272 MBytes 2.28 Gbits/sec
[ 7] 4.00-5.00 sec 271 MBytes 2.28 Gbits/sec
[ 7] 5.00-6.00 sec 271 MBytes 2.27 Gbits/sec
[ 7] 6.00-7.00 sec 272 MBytes 2.28 Gbits/sec
[ 7] 7.00-8.00 sec 271 MBytes 2.28 Gbits/sec
[ 7] 8.00-9.00 sec 268 MBytes 2.25 Gbits/sec
[ 7] 9.00-10.00 sec 269 MBytes 2.26 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 7] 0.00-10.00 sec 2.55 GBytes 2.19 Gbits/sec sender
[ 7] 0.00-10.01 sec 2.55 GBytes 2.19 Gbits/sec receiver |
nirs
reviewed
Oct 14, 2025
| } | ||
| krunCmd.SysProcAttr = executil.BackgroundSysProcAttr | ||
|
|
||
| logPath := filepath.Join(l.Instance.Dir, "krun.log") |
There was a problem hiding this comment.
"krunkit.log" to match the driver name.
|
The default mode shouldn't depend on |
Should I try with gvisor? |
Yes |
|
I think this can work similar to vz:
In both case krunkit will use virtio-net,type=unixstream,mac=,...path=... |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Important
This PR is not yet ready for reviewing. Networking and file sharing is yet to be implemented. Many things are also hardcoded directly and that needs to be fixed too.