Performance regression with macOS > 14.7

[nirs]

Running performance tests with macOS 15.1, 15.1.1, and 15.2 show 35% regression compared with 14.7.1.

host to vm

vz

% iperf3-darwin -c 192.168.105.12 -l 1m      
Connecting to host 192.168.105.12, port 5201
[  5] local 192.168.105.1 port 51210 connected to 192.168.105.12 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd          RTT
[  5]   0.00-1.00   sec   271 MBytes  2.27 Gbits/sec    0   8.00 MBytes   14ms     
[  5]   1.00-2.00   sec   265 MBytes  2.22 Gbits/sec    0   8.00 MBytes   15ms     
[  5]   2.00-3.00   sec   268 MBytes  2.25 Gbits/sec    0   8.00 MBytes   16ms     
[  5]   3.00-4.00   sec   271 MBytes  2.28 Gbits/sec    0   8.00 MBytes   13ms     
[  5]   4.00-5.00   sec   272 MBytes  2.28 Gbits/sec    0   8.00 MBytes   14ms     
[  5]   5.00-6.00   sec   270 MBytes  2.26 Gbits/sec    0   8.00 MBytes   15ms     
[  5]   6.00-7.00   sec   271 MBytes  2.27 Gbits/sec    0   8.00 MBytes   14ms     
[  5]   7.00-8.00   sec   271 MBytes  2.27 Gbits/sec    0   8.00 MBytes   15ms     
[  5]   8.00-9.00   sec   272 MBytes  2.28 Gbits/sec    0   8.00 MBytes   14ms     
[  5]   9.00-10.00  sec   268 MBytes  2.25 Gbits/sec    0   8.00 MBytes   14ms     
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  2.63 GBytes  2.26 Gbits/sec    0             sender
[  5]   0.00-10.23  sec  2.63 GBytes  2.21 Gbits/sec                  receiver

Similar run on macOS 14.7, copied from #58:

% caffeinate -d iperf3-darwin -c 192.168.105.58 -l 1m -t 10
Connecting to host 192.168.105.58, port 5201
[  5] local 192.168.105.1 port 60990 connected to 192.168.105.58 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd          RTT
[  5]   0.00-1.00   sec   460 MBytes  3.86 Gbits/sec    0   8.00 MBytes   9ms     
[  5]   1.00-2.00   sec   421 MBytes  3.53 Gbits/sec    0   8.00 MBytes   9ms     
[  5]   2.00-3.00   sec   435 MBytes  3.65 Gbits/sec    0   8.00 MBytes   10ms     
[  5]   3.00-4.00   sec   411 MBytes  3.45 Gbits/sec    0   8.00 MBytes   14ms     
[  5]   4.00-5.00   sec   317 MBytes  2.66 Gbits/sec    0   8.00 MBytes   9ms     
[  5]   5.00-6.00   sec   430 MBytes  3.61 Gbits/sec    0   8.00 MBytes   9ms     
[  5]   6.00-7.00   sec   423 MBytes  3.55 Gbits/sec    0   8.00 MBytes   9ms     
[  5]   7.00-8.00   sec   433 MBytes  3.63 Gbits/sec    0   8.00 MBytes   10ms     
[  5]   8.00-9.00   sec   437 MBytes  3.67 Gbits/sec    0   8.00 MBytes   9ms     
[  5]   9.00-10.00  sec   430 MBytes  3.61 Gbits/sec    0   8.00 MBytes   9ms     
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  4.10 GBytes  3.52 Gbits/sec    0             sender
[  5]   0.00-10.00  sec  4.10 GBytes  3.52 Gbits/sec                  receiver

qemu

For reference, same test with qemu instead of vz on macOS 15.2:

% iperf3-darwin -c 192.168.105.13 -l 1m
Connecting to host 192.168.105.13, port 5201
[  5] local 192.168.105.1 port 51254 connected to 192.168.105.13 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd          RTT
[  5]   0.00-1.00   sec   205 MBytes  1.72 Gbits/sec    0   6.36 MBytes   15ms     
[  5]   1.00-2.00   sec   206 MBytes  1.73 Gbits/sec    0   6.36 MBytes   15ms     
[  5]   2.00-3.00   sec   209 MBytes  1.75 Gbits/sec    0   6.36 MBytes   16ms     
[  5]   3.00-4.00   sec   211 MBytes  1.77 Gbits/sec    0   6.36 MBytes   15ms     
[  5]   4.00-5.00   sec   204 MBytes  1.71 Gbits/sec    0   6.36 MBytes   15ms     
[  5]   5.00-6.00   sec   196 MBytes  1.64 Gbits/sec    0   6.36 MBytes   15ms     
[  5]   6.00-7.00   sec   203 MBytes  1.70 Gbits/sec    0   6.36 MBytes   15ms     
[  5]   7.00-8.00   sec   203 MBytes  1.70 Gbits/sec    0   6.36 MBytes   13ms     
[  5]   8.00-9.00   sec   207 MBytes  1.74 Gbits/sec    0   6.36 MBytes   19ms     
[  5]   9.00-10.00  sec   205 MBytes  1.72 Gbits/sec    0   6.36 MBytes   15ms     
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  2.00 GBytes  1.72 Gbits/sec    0             sender
[  5]   0.00-10.01  sec  2.00 GBytes  1.72 Gbits/sec                  receiver

I don't have qemu results from macOS 14.7.

vm to vm

macOS 15.2:

$ iperf3 -c 192.168.105.12 -l 1m
Connecting to host 192.168.105.12, port 5201
[  5] local 192.168.105.14 port 45308 connected to 192.168.105.12 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   141 MBytes  1.18 Gbits/sec    0   3.91 MBytes       
[  5]   1.00-2.00   sec   141 MBytes  1.18 Gbits/sec    0   3.91 MBytes       
[  5]   2.00-3.00   sec   135 MBytes  1.13 Gbits/sec    0   3.91 MBytes       
[  5]   3.00-4.00   sec   135 MBytes  1.13 Gbits/sec    0   3.91 MBytes       
[  5]   4.00-5.00   sec   142 MBytes  1.19 Gbits/sec    0   3.91 MBytes       
[  5]   5.00-6.00   sec   134 MBytes  1.12 Gbits/sec    0   3.91 MBytes       
[  5]   6.00-7.00   sec   127 MBytes  1.07 Gbits/sec    0   3.91 MBytes       
[  5]   7.00-8.00   sec   139 MBytes  1.16 Gbits/sec    0   3.91 MBytes       
[  5]   8.00-9.00   sec   137 MBytes  1.15 Gbits/sec    0   3.91 MBytes       
[  5]   9.00-10.00  sec   139 MBytes  1.17 Gbits/sec    0   3.91 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.34 GBytes  1.15 Gbits/sec    0             sender
[  5]   0.00-10.02  sec  1.34 GBytes  1.15 Gbits/sec                  receiver

I don't have results from 14.7, but I remember if was around 2 Gbits/s.

More info

Tested with both launchd service and by running socket_vment manually in a shell, so eliminate ProcessType regression in launched.

Tested on both managed mac with corporate VPN and content filters, and unmanned mac, both show the same regression.

Testing forwarding packets from vmnet using vfkit does not show any regression, so the issue is not in vmnet and likely not in socket_vment. It may be a performance regression in Go on macOS 15, affecting forwarding packets in lima.

I eliminated issues with not using the mac address assigned by vment, by testing vfkit with vment using both assigned mac addreess and unrelated mac address. Both show the same performance with iperf3.

I think we need a benchmark tool sending and receiving qemu packets to socket_vment unix socket. This will make it easy to tell if a performance regression is in socket_vment, lima, or qemu.