VMWare、Hyper-V、iperf 和嵌套虚拟化
发表于|更新于
|总字数:20k|阅读时长:1:11分钟
VMWare Workstation 中创建的虚拟机默认采用 e1000 或 pcnet32 网卡：
1
2
3
4
5
6
7
8
9
10
11
| arrio@ubuntu1 ~> ethtool -i ens33
driver: pcnet32
version: 6.17.0-8-generic
firmware-version:
expansion-rom-version:
bus-info: 0000:02:01.0
supports-statistics: no
supports-test: yes
supports-eeprom-access: no
supports-register-dump: yes
supports-priv-flags: no
|
它们的性能表现并不理想。尤其是在开启了 Hyper-V 的情况下，iperf 测速能达到 100Mbps 就不错了：
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
| ❯ iperf3 -c 172.31.28.101
Connecting to host 172.31.28.101, port 5201
[ 5] local 172.31.28.1 port 54451 connected to 172.31.28.101 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.01 sec 9.38 MBytes 78.2 Mbits/sec
[ 5] 1.01-2.01 sec 9.75 MBytes 81.4 Mbits/sec
[ 5] 2.01-3.01 sec 9.75 MBytes 82.1 Mbits/sec
[ 5] 3.01-4.00 sec 9.88 MBytes 83.0 Mbits/sec
[ 5] 4.00-5.01 sec 9.88 MBytes 82.3 Mbits/sec
[ 5] 5.01-6.01 sec 10.1 MBytes 85.4 Mbits/sec
[ 5] 6.01-7.00 sec 9.88 MBytes 83.2 Mbits/sec
[ 5] 7.00-8.01 sec 10.1 MBytes 84.5 Mbits/sec
[ 5] 8.01-9.01 sec 10.5 MBytes 88.2 Mbits/sec
[ 5] 9.01-10.01 sec 10.4 MBytes 86.7 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.01 sec 99.6 MBytes 83.5 Mbits/sec sender
[ 5] 0.00-10.01 sec 99.4 MBytes 83.3 Mbits/sec receiver
iperf Done.
|
如果需要更高的性能，可以使用 vmxnet3 类型的网卡。它可以将速度提升至大约 3Gbps：
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
| ❯ cat Ubuntu.vmx
...
ethernet0.virtualDev = "vmxnet3"
...
❯ iperf3 -c 172.31.27.101
Connecting to host 172.31.27.101, port 5201
[ 5] local 172.31.27.1 port 31000 connected to 172.31.27.101 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 340 MBytes 2.84 Gbits/sec
[ 5] 1.00-2.01 sec 325 MBytes 2.70 Gbits/sec
[ 5] 2.01-3.01 sec 339 MBytes 2.85 Gbits/sec
[ 5] 3.01-4.00 sec 314 MBytes 2.66 Gbits/sec
[ 5] 4.00-5.01 sec 332 MBytes 2.77 Gbits/sec
[ 5] 5.01-6.01 sec 349 MBytes 2.93 Gbits/sec
[ 5] 6.01-7.01 sec 348 MBytes 2.91 Gbits/sec
[ 5] 7.01-8.00 sec 361 MBytes 3.06 Gbits/sec
[ 5] 8.00-9.01 sec 336 MBytes 2.80 Gbits/sec
[ 5] 9.01-10.01 sec 359 MBytes 3.02 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.01 sec 3.32 GBytes 2.85 Gbits/sec sender
[ 5] 0.00-10.02 sec 3.32 GBytes 2.85 Gbits/sec receiver
iperf Done.
|
尽管如此，这个速度仍然算不上令人满意。要知道这可是正经的 企业级网卡，在 vSphere 中，它的链路速度甚至可以达到 65Gbps。
那么为什么在 Workstation 中性能会大打折扣呢？一个可能的原因是在开启 Hyper-V 之后，VMWare 实际上转变为 Type 2 类型的 hypervisor，对硬件的访问必须经过 WHP (Windows Hypervisor Platform)，从而大幅降低了 I/O 效率。具体的工作模式可以在 vmware.log 日志文件中查看：
1
2
3
4
5
| ❯ cat vmware.log | rg "Monitor Mode"
2026-01-13T07:50:49.689Z In(05) vmx Monitor Mode: ULM
# CPL0: Current Privilege Level 0, high performance
# ULM: User-Level Monitor, nested virtualization(?
# see also: https://blog.syselement.com/home/home-lab/hypervisors/vmware/vmware-workstation
|
直接关闭 Hyper-V 并不是一个理想的解决方案，因为 WSL 以及 Windows 的某些内核功能（比如内存完整性等）都依赖于 Hyper-V。
更好的做法是直接使用 Hyper-V 作为 hypervisor 平台。在这种模式下，iperf 网络测试能够轻松跑出单线程 5Gbps、多线程 10Gbps 的速度：
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
| ❯ iperf3 -c 172.28.195.175
Connecting to host 172.28.195.175, port 5201
[ 5] local 172.28.192.1 port 18999 connected to 172.28.195.175 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.01 sec 487 MBytes 4.07 Gbits/sec
[ 5] 1.01-2.00 sec 682 MBytes 5.72 Gbits/sec
[ 5] 2.00-3.00 sec 686 MBytes 5.76 Gbits/sec
[ 5] 3.00-4.01 sec 682 MBytes 5.68 Gbits/sec
[ 5] 4.01-5.01 sec 705 MBytes 5.91 Gbits/sec
[ 5] 5.01-6.01 sec 639 MBytes 5.37 Gbits/sec
[ 5] 6.01-7.01 sec 641 MBytes 5.39 Gbits/sec
[ 5] 7.01-8.01 sec 690 MBytes 5.80 Gbits/sec
[ 5] 8.01-9.01 sec 684 MBytes 5.72 Gbits/sec
[ 5] 9.01-10.01 sec 628 MBytes 5.29 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.01 sec 6.37 GBytes 5.47 Gbits/sec sender
[ 5] 0.00-10.01 sec 6.37 GBytes 5.47 Gbits/sec receiver
iperf Done.
❯ iperf3 -c 172.28.195.175 -P 4
Connecting to host 172.28.195.175, port 5201
[ 5] local 172.28.192.1 port 18988 connected to 172.28.195.175 port 5201
[ 7] local 172.28.192.1 port 18989 connected to 172.28.195.175 port 5201
[ 9] local 172.28.192.1 port 18990 connected to 172.28.195.175 port 5201
[ 11] local 172.28.192.1 port 18991 connected to 172.28.195.175 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.01 sec 353 MBytes 2.93 Gbits/sec
[ 7] 0.00-1.01 sec 341 MBytes 2.83 Gbits/sec
[ 9] 0.00-1.01 sec 260 MBytes 2.16 Gbits/sec
[ 11] 0.00-1.01 sec 297 MBytes 2.46 Gbits/sec
[SUM] 0.00-1.01 sec 1.22 GBytes 10.4 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 1.01-2.01 sec 308 MBytes 2.59 Gbits/sec
[ 7] 1.01-2.01 sec 305 MBytes 2.57 Gbits/sec
[ 9] 1.01-2.01 sec 305 MBytes 2.57 Gbits/sec
[ 11] 1.01-2.01 sec 294 MBytes 2.48 Gbits/sec
[SUM] 1.01-2.01 sec 1.18 GBytes 10.2 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 2.01-3.01 sec 338 MBytes 2.82 Gbits/sec
[ 7] 2.01-3.01 sec 323 MBytes 2.70 Gbits/sec
[ 9] 2.01-3.01 sec 337 MBytes 2.82 Gbits/sec
[ 11] 2.01-3.01 sec 326 MBytes 2.72 Gbits/sec
[SUM] 2.01-3.01 sec 1.29 GBytes 11.1 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 3.01-4.01 sec 302 MBytes 2.53 Gbits/sec
[ 7] 3.01-4.01 sec 314 MBytes 2.63 Gbits/sec
[ 9] 3.01-4.01 sec 310 MBytes 2.60 Gbits/sec
[ 11] 3.01-4.01 sec 301 MBytes 2.53 Gbits/sec
[SUM] 3.01-4.01 sec 1.20 GBytes 10.3 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 4.01-5.01 sec 384 MBytes 3.23 Gbits/sec
[ 7] 4.01-5.01 sec 362 MBytes 3.04 Gbits/sec
[ 9] 4.01-5.01 sec 377 MBytes 3.17 Gbits/sec
[ 11] 4.01-5.01 sec 352 MBytes 2.95 Gbits/sec
[SUM] 4.01-5.01 sec 1.44 GBytes 12.4 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 5.01-6.01 sec 348 MBytes 2.91 Gbits/sec
[ 7] 5.01-6.01 sec 354 MBytes 2.96 Gbits/sec
[ 9] 5.01-6.01 sec 311 MBytes 2.60 Gbits/sec
[ 11] 5.01-6.01 sec 295 MBytes 2.47 Gbits/sec
[SUM] 5.01-6.01 sec 1.28 GBytes 10.9 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 6.01-7.01 sec 294 MBytes 2.48 Gbits/sec
[ 7] 6.01-7.01 sec 297 MBytes 2.50 Gbits/sec
[ 9] 6.01-7.01 sec 305 MBytes 2.57 Gbits/sec
[ 11] 6.01-7.01 sec 297 MBytes 2.50 Gbits/sec
[SUM] 6.01-7.01 sec 1.16 GBytes 10.0 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 7.01-8.01 sec 323 MBytes 2.69 Gbits/sec
[ 7] 7.01-8.01 sec 322 MBytes 2.69 Gbits/sec
[ 9] 7.01-8.01 sec 322 MBytes 2.69 Gbits/sec
[ 11] 7.01-8.01 sec 314 MBytes 2.62 Gbits/sec
[SUM] 7.01-8.01 sec 1.25 GBytes 10.7 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 8.01-9.01 sec 326 MBytes 2.74 Gbits/sec
[ 7] 8.01-9.01 sec 324 MBytes 2.71 Gbits/sec
[ 9] 8.01-9.01 sec 301 MBytes 2.53 Gbits/sec
[ 11] 8.01-9.01 sec 333 MBytes 2.80 Gbits/sec
[SUM] 8.01-9.01 sec 1.25 GBytes 10.8 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 9.01-10.00 sec 229 MBytes 1.94 Gbits/sec
[ 7] 9.01-10.00 sec 189 MBytes 1.60 Gbits/sec
[ 9] 9.01-10.00 sec 230 MBytes 1.95 Gbits/sec
[ 11] 9.01-10.00 sec 211 MBytes 1.79 Gbits/sec
[SUM] 9.01-10.00 sec 859 MBytes 7.30 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.00 sec 3.13 GBytes 2.69 Gbits/sec sender
[ 5] 0.00-10.00 sec 3.13 GBytes 2.69 Gbits/sec receiver
[ 7] 0.00-10.00 sec 3.06 GBytes 2.62 Gbits/sec sender
[ 7] 0.00-10.00 sec 3.06 GBytes 2.62 Gbits/sec receiver
[ 9] 0.00-10.00 sec 2.99 GBytes 2.57 Gbits/sec sender
[ 9] 0.00-10.00 sec 2.99 GBytes 2.57 Gbits/sec receiver
[ 11] 0.00-10.00 sec 2.95 GBytes 2.53 Gbits/sec sender
[ 11] 0.00-10.00 sec 2.95 GBytes 2.53 Gbits/sec receiver
[SUM] 0.00-10.00 sec 12.1 GBytes 10.4 Gbits/sec sender
[SUM] 0.00-10.00 sec 12.1 GBytes 10.4 Gbits/sec receiver
iperf Done.
|
虚拟化厂商常宣传“性能损耗不到 5%”，但这通常是指用户态计算密集型的理想情况。一旦面对用户态/内核态切换、高中断频率或者 IO 密集的任务，性能往往会出现显著下降。虽然 VT-x/AMD-V 等技术进一步降低了虚拟化开销，但在嵌套虚拟化环境下，性能表现仍难以令人满意。
在 Windows 上，如果已经启用了 Hyper-V，最好将所有的虚拟机全部迁移到 Hyper-V 运行。这样可以充分利用 Hyper-V 的 Type 1 hypervisor 的性能优势，提高虚拟化效率。
