KVM Lab Setup
Qemu stuff I did before settingup KVM
sudo apt install qemu sudo apt install qemu-utils sudo apt install qemu-system-x86
qemu-img create ubuntu.img 20G
qemu-system-x86_64 -hda ubuntu.img -boot d -cdrom /home/user1/Downloads/rhel-8.4-x86_64-dvd.iso -m 1000 qemu-system-x86_64 -hda ubuntu.img -boot d -cdrom /home/user1/Downloads/rhel-8.4-x86_64-dvd.iso -m 1000
sudo apt install libvirt-clients virsh
KVM
egrep -c ‘(vmx | svm)’ /proc/cpuinfo |
0 # cpu doesn’t support hardware virtualization 1 or more # cpu does support hardware virtualization ( still needs to be enabled in bios )
Check if it is actually enabled ( Xen kernel only ):
cat /sys/hypervisor/properties/capabilities
Also check if it is enabled with this: sudo apt update sudo apt install cpu-checker # probably already there kvm-ok
sudo apt-get install qemu-kvm libvirt-daemon-system libvirt-clients bridge-utils virtinst sudo apt-get install virt-manager
sudo systemctl is-active libvirtd
sudo usermod -aG libvirt $USER sudo usermod -aG kvm $USER
virsh list –all
brctl show
“Virtual Machine Manager” - this is virt-manager
Serial Console:
virsh ttyconsole my_vm more info: https://help.ubuntu.com/community/KVM/Access
Networking: https://help.ubuntu.com/community/KVM/Networking#Bridged_Networking
https://levelup.gitconnected.com/how-to-setup-bridge-networking-with-kvm-on-ubuntu-20-04-9c560b3e3991 <======================
sudo brctl addbr br0
If you have any issues:
if any error - log out and back in ( from terminal or also from desktop especially if launching virt-manager from a menu )
- check if you can write to this
sudo ls -la /var/run/libvirt/libvirt-sock
If you have problems creating a VM:
sudo chown root:libvirtd /dev/kvm
relogin or restart kernel modules
lsmod | grep -i kvm |
rmmod kvm modprobe -a kvm
virsh –connect qemu:///system create alice.xml # might be needed if you have another hypervisor like VirtualBox
https://www.xmodulo.com/use-kvm-command-line-debian-ubuntu.html
qemu-img create -f qcow2 alice.img 10G ls -l Downloads/ubuntu-20.04.3-live-server-amd64.iso vi alice.xml virsh create alice.xml virsh start alice virsh list sudo brctl addbr br0 sudo brctl show
sudo netstat -nap | egrep ‘(kvm|qemu)’ # find vnc port virsh destroy alice virsh shutdown alice virsh suspend alice virsh resume alice virsh console alice virsh autostart alice virsh dominfo alice virsh edit alice virsh undefine alice # actually delete vm ( shutdown before or after )
virsh domrename vm1 template # rename a VM
virsh # basically run a virsh shell
==================================================
disable netfilter on bridges
/etc/sysctl.d/bridge.conf
net.bridge.bridge-nf-call-ip6tables=0 net.bridge.bridge-nf-call-iptables=0 net.bridge.bridge-nf-call-arptables=0
/etc/udev/rules.d/99-bridge.rules
ACTION==”add”, SUBSYSTEM==”module”, KERNEL==”br_netfilter”, \ RUN+=”/sbin/sysctl -p /etc/sysctl.d/bridge.conf”
remove default kvm interfaces:
virsh net-destroy default virsh net-undefine default
remove with these commands if the interfaces haven’t been removed:
ip link delete virbr0 type brigde ip link delete virbr0-nic.
/etc/netplan/00-installer-config.yaml
network: ethernets: enp0s7: dhcp4: false dhcp6: false bridges: br0: interfaces: [ enp0s7 ] addresses: [192.168.0.104/24] gateway4: 192.168.0.1 mtu: 1500 nameservers: addresses: [8.8.8.8,8.8.4.4] parameters: stp: true forward-delay: 4 dhcp4: no dhcp6: no version: 2
network: ethernets: eno2: dhcp4: false dhcp6: false bridges: br0: interfaces: [ eno2 ] addresses: [192.168.3.22/24] gateway4: 192.168.3.1 mtu: 1500 nameservers: addresses: [8.8.8.8,8.8.4.4] parameters: stp: true forward-delay: 4 dhcp4: no dhcp6: no version: 2
Above, static config from enp0s7 is moved to br0
sudo netplan apply
host-bridge.xml
virsh net-define host-bridge.xml virsh net-start host-bridge virsh net-autostart host-bridge
virsh net-list –all
Change interface for dhcp server:
sudo vi /etc/default/isc-dhcp-server
INTERFACESv4=”br0”
example: virt-install –name vm1 –ram=8192 –disk size=10 –vcpus 1 –os-type linux –os-variant ubuntu20.04 –graphics none –location ‘http://archive.ubuntu.com/ubuntu/dists/focal/main/installer-amd64/’ –extra-args “console=tty0 console=ttyS0,115200n8” working: virt-install –name vm1 –ram=8192 –disk size=10 –vcpus 1 –os-type linux –os-variant ubuntu20.04 –cdrom /home/user1/Downloads/ubuntu-20.04.3-live-server-amd64.iso
- or just create VM from GUI
virsh domifaddr vm1 # get ip address ssh-copy-id rkamradt@192.168.122.95 # use ip address from previous step
virsh domifaddr node1
make sure your guest VM requests a DHCP lease by MAC and not hostid
/etc/netplan/00-installer-config.yaml
This is the network config written by ‘subiquity’
network: ethernets: enp1s0: dhcp4: true dhcp-identifier: mac version: 2
Cloning VMs
virt-clone –original vm1 –name vm2 –auto-clone virt-clone –original vm1 –name vm3 –auto-clone
Create 10 clones:
for i in 0 1 2 3 4 5 6 7 8 9; do virt-clone –original template –name vm$i –auto-clone; done
Get MAC address:
virsh domiflist vm1
virsh domiflist vm5|grep -i bridge| awk ‘{ print $5 }’
for i in virsh list --all| tail -n +3| head -n -1 | awk '{print $2}'
; do virsh domiflist $i|grep -i bridge| awk ‘{ print $5 }’; done ## list all MACs
Start all VMs:
for i in virsh list --all| tail -n +3| head -n -1 | awk '{print $2}'
; do virsh start $i; done
Stop all VMs:
for i in virsh list --all| tail -n +3| head -n -1 | awk '{print $2}'
; do virsh stop $i; done
for i in virsh list --all| tail -n +3| head -n -1 | awk '{print $2}' ; do virsh domiflist $i |
grep -i bridge | awk ‘{ print “ host test1 {“ $5 }’; done ## list all MACs |
host test1 { hardware ethernet 08:00:27:85:AE:C9; fixed-address 192.168.3.150; }
Map IPs to MACs in DHCP:
Almost except that 10 is wrong, need better counting. Use IP pools in Python: ( also maybe exclude the template and create a pool for names )
n=0;echo $n;for i in virsh list --all| tail -n +3| head -n -1 | awk '{print $2}' ; do echo “host test$n { hardware ethernet “ virsh domiflist $i | grep -i bridge | awk '{ print $5}' “; fixed-address 192.168.3.15$n; }”; let n=n+1; done |
sed ‘s/ ;/;/’ |
Don’t really need this once it is added to the template:
cat ~/.ssh/id_rsa.pub | ssh user1@hostname ‘cat » .ssh/authorized_keys’ |
OR
ssh-copy-id -i ~/.ssh/id_rsa.pub user1@host
for i in 51 52 53 54 55 56 57 58 59 60; do ssh-copy-id -i ~/.ssh/id_rsa.pub user1@192.168.3.1$i; done