First of all, please enable hardware virtualization and VT-d (I/O virtualization) in BIOS. It's mandatory!
List all available NIC's on the server:
If you got: "error: Operation not supported: neither VFIO nor KVM device assignment is currently supported on this system".
Well, we should enable IOMMU for fixing this issue:
Sources: https://www.paloaltonetworks.com/content/dam/paloaltonetworks-com/en_US/assets/pdf/framemaker/61/pan-os/NewFeaturesGuide/section_7.pdf and http://h20564.www2.hp.com/hpsc/doc/public/display?docId=emr_na-c03645796
Install KVM tools and distro install tools on the Debian:
apt-get install -y qemu-kvm libvirt-daemon libvirt-clients libvirt-daemon-system virtinstEnable libvirt and libvirt.guests services:
systemctl restart libvirtd.serviceRun default network:
systemctl restart libvirt-guests.service
virsh net-autostart defaultInstall Debian Jessie:
virsh net-start default
virt-install --location http://ftp.us.debian.org/debian/dists/stable/main/installer-amd64/ --ram 16000 --file /var/lib/libvirt/images/fastnetmonvm.fastvps.ru.disk --file-size 20 --name fastnetmonvm.fastvps.ru -x console=ttyS0Switch on autostart for VM:
virsh autostart fastnetmonvm.fastvps.ruWell, we should pass whole NIC to VM.
List all available NIC's on the server:
lspci |grep EtherWe will pass two NICs (it's two port NIC card):
03:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
03:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
07:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
03:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)Get PCI device names in virsh format (03_00_0 created from 03:00.0 with changing ':' by '_'):
03:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
virsh nodedev-list | grep pci |grep 03_00_0Get XML descriptions for both PCI devices:
pci_0000_03_00_0
pci_0000_03_00_1
virsh nodedev-dumpxml pci_0000_03_00_0Switch off drivers for this NIC's on server:
virsh nodedev-dumpxml pci_0000_03_00_1
ifconfig ethX down
rmmod ixgbe
Detach both NIC from the Linux:
virsh nodedev-dettach pci_0000_03_00_0
virsh nodedev-dettach pci_0000_03_00_1
Well, we should enable IOMMU for fixing this issue:
vim /etc/default/grubChange following line to:
GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on"Apply grub settings:
update-grub
And reboot server:
shutdown -r now
Then repeat last steps.
List devices from virsh:
virsh nodedev-dumpxml pci_0000_03_00_0 |grep address
<address domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
<address domain='0x0000' bus='0x03' slot='0x00' function='0x1'/>
Then please build something like this using data above:
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x0a' slot='0x00' function='0x0'/>
</source>
</hostdev>
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x0a' slot='0x00' function='0x1'/>
</source>
</hostdev>
Then open VM edit and insert this block to "devices" block:
virsh edit fastnetmonvm.fastvps.ru
Then you could try to run VM:
virsh start fastnetmonvm.fastvps.ru
You could get error similar to this:
error: Failed to start domain fastnetmonvm.fastvps.ru
error: internal error: early end of file from monitor: possible problem:
2015-05-13T13:12:39.353614Z qemu-system-x86_64: -device vfio-pci,host=0a:00.0,id=hostdev0,bus=pci.0,addr=0x6: vfio: failed to set iommu for container: Operation not permitted
2015-05-13T13:12:39.353761Z qemu-system-x86_64: -device vfio-pci,host=0a:00.0,id=hostdev0,bus=pci.0,addr=0x6: vfio: failed to setup container for group 23
2015-05-13T13:12:39.353780Z qemu-system-x86_64: -device vfio-pci,host=0a:00.0,id=hostdev0,bus=pci.0,addr=0x6: vfio: failed to get group 23
2015-05-13T13:12:39.353804Z qemu-system-x86_64: -device vfio-pci,host=0a:00.0,id=hostdev0,bus=pci.0,addr=0x6: Device initialization failed.
2015-05-13T13:12:39.353827Z qemu-system-x86_64: -device vfio-pci,host=0a:00.0,id=hostdev0,bus=pci.0,addr=0x6: Device 'vfio-pci' could not be initialized
If you have something like " vfio-pci 0000:0a:00.0: Device is ineligible for IOMMU domain attach due to platform RMRR requirement. Contact your platform vendor." in dmesg that's not a good news for you. You could not use IOMMU for your server and need to buy another.
For fix issue "vfio_iommu_type1_attach_group: No interrupt remapping support" please use:
echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/vfio_iommu_type1.confAnd reboot again.
Sources: https://www.paloaltonetworks.com/content/dam/paloaltonetworks-com/en_US/assets/pdf/framemaker/61/pan-os/NewFeaturesGuide/section_7.pdf and http://h20564.www2.hp.com/hpsc/doc/public/display?docId=emr_na-c03645796
Thanks for sharing, nice post! Post really provice useful information!
ReplyDeleteGiaonhan247 chuyên dịch vụ mua hộ hàng hàn quốc và dịch vụ vận chuyển hàng trung quốc về việt nam giá rẻ hay dịch vụ order hàng đức và ship hàng đức về việt nam về VN uy tín, hay dịch vụ mua hàng trên amazon có an toàn không cũng như mua đồng hồ trên amazon nhật có đảm bảo không uy tín không.