[spp] [PATCH 01/30] doc: add setup guide
Yasufumi Ogawa
ogawa.yasufumi at lab.ntt.co.jp
Fri Jan 19 01:52:19 CET 2018
On 2018/01/16 14:16, x-fn-spp at sl.ntt-tx.co.jp wrote:
> From: Hiroyuki Nakamura <nakamura.hioryuki at po.ntt-tx.co.jp>
>
> Hi all,
>
>> Series applied, thanks.
>
> Ferruh, thank you for accepting our patches.
>
> We will post the document patches for spp_vf on DPDK17.08 in the
> following emails.
>
>
> Translate Japanese setup guide to English.
>
> Signed-off-by: Naoki Takada <takada.naoki at lab.ntt.co.jp>
I tried to apply your patches but failed because of an error in 8th
patch as following. Could you fix it? Thanks.
Applying: doc: modify figure in spp_vf_overview
error: patch failed: docs/spp_vf/spp_vf_overview.svg:215
error: docs/spp_vf/spp_vf_overview.svg: patch does not apply
Patch failed at 0008 doc: modify figure in spp_vf_overview
Yasufumi
> ---
> docs/spp_vf/setup.md | 288 +++++++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 288 insertions(+)
> create mode 100644 docs/spp_vf/setup.md
>
> diff --git a/docs/spp_vf/setup.md b/docs/spp_vf/setup.md
> new file mode 100644
> index 0000000..572777b
> --- /dev/null
> +++ b/docs/spp_vf/setup.md
> @@ -0,0 +1,288 @@
> +# Setup Guide
> +
> +## Environment
> +
> +* Ubuntu 16.04
> +* qemu-kvm
> +* DPDK v17.05
> +
> +## Setting
> +
> +### Host
> +
> +#### Edit Config
> +
> +Uncomment user and group in `/etc/libvirt/qemu.conf`.
> +
> +```sh
> +# /etc/libvirt/qemu.conf
> +
> +user = "root"
> +group = "root"
> +```
> +
> +Change `KVM_HUGEPAGES` from 0 to 1 in `/etc/default/qemu-kvm`.
> +
> +```sh
> +# /etc/default/qemu-kvm
> +
> +KVM_HUGEPAGES=1
> +```
> +
> +Change grub config for hugepages and isolcpus.
> +
> +```sh
> +# /etc/default/grub
> +
> +GRUB_CMDLINE_LINUX_DEFAULT="isolcpus=2,4,6,8,10,12-18,20,22,24,26-42,44,46 hugepagesz=1G hugepages=36 default_hugepagesz=1G"
> +```
> +
> +You need to run `update-grub` and reboot to activate grub config.
> +
> +```sh
> +$ sudo upadte-grub
> +$ sudo reboot
> +```
> +
> +You can check hugepage settings as following.
> +
> +```sh
> +$ cat /proc/meminfo | grep -i huge
> +AnonHugePages: 2048 kB
> +HugePages_Total: 36 # /etc/default/grub
> +HugePages_Free: 36
> +HugePages_Rsvd: 0
> +HugePages_Surp: 0
> +Hugepagesize: 1048576 kB # /etc/default/grub
> +
> +$ mount | grep -i huge
> +cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb,release_agent=/run/cgmanager/agents/cgm-release-agent.hugetlb,nsroot=/)
> +hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
> +hugetlbfs-kvm on /run/hugepages/kvm type hugetlbfs (rw,relatime,mode=775,gid=117)
> +hugetlb on /run/lxcfs/controllers/hugetlb type cgroup (rw,relatime,hugetlb,release_agent=/run/cgmanager/agents/cgm-release-agent.hugetlb,nsroot=/)
> +```
> +
> +Finally, you unmount default hugepage.
> +
> +```sh
> +$ sudo unmount /dev/hugepages
> +```
> +
> +#### Install jasson
> +
> +Network configuration is defined in JSON and `spp_vf` reads config from
> +the file while launching.
> +[jasson](http://www.digip.org/jansson/) is a JSON library written in C.
> +
> +It is required to use `json_path` feature of `jasson` for `spp_vf`.
> +It has develped under `json_path` branch and you need to checkout and compile
> +it manually.
> +
> +```sh
> +$ git clone https://github.com/rogerz/jansson
> +$ cd jansson
> +$ sudo git checkout json_path
> +Branch json_path set up to track remote branch json_path from origin. Switched to a new branch 'json_path'
> +```
> +
> +This setup guide expects that `jasson` is placed as `/opt/jasson`.
> +
> +```sh
> +$ sudo mkdir -p /opt/jansson
> +$ sudo mv jansson /opt/jansson
> +```
> +
> +Compile it as following.
> +
> +```sh
> +$ cd /opt/jansson/jansson
> +$ sudo autoreconf -i
> +$ sudo ./configure
> +$ sudo make
> +$ sudo make install
> +$ sudo ldconfig
> +```
> +
> +Then, confirm that header files of jasson are generated in `/usr/local/include`.
> +
> +```sh
> +$ ls -al /usr/local/include
> +total 24
> +drwxr-xr-x 2 root root 4096 Jul 28 16:45 .
> +drwxr-xr-x 10 root root 4096 May 27 10:23 ..
> +-rw-r--r-- 1 root root 1183 Jul 28 16:45 jansson_config.h
> +-rw-r--r-- 1 root root 9499 Jul 28 16:45 jansson.h
> +```
> +
> +#### Install DPDK
> +
> +Install DPDK v17.05 in any directory. This is a simple instruction and please refer
> +[Getting Started Guide for Linux](http://dpdk.org/doc/guides/linux_gsg/index.html)
> +for details.
> +
> +```sh
> +$ cd /path/to/any_dir
> +$ git clone http://dpdk.org/git/dpdk
> +$ cd dpdk
> +$ git checkout v17.05
> +$ export RTE_SDK=`pwd`
> +$ export RTE_TARGET=x86_64-native-linuxapp-gcc
> +$ make T=x86_64-native-linuxapp-gcc install
> +```
> +
> +#### Install SPP
> +
> +Clone SPP in any directory and compile it.
> +
> +```sh
> +$ cd /path/to/spp_home/
> +$ git clone https://github.com/ntt-ns/Soft-Patch-Panel.git
> +export SPP_HOME=/path/to/spp_home/Soft-Patch-Panel
> +$ cd $SPP_HOME
> +$ make
> +```
> +
> +#### Setup for DPDK
> +
> +Load igb_uio module.
> +
> +```sh
> +$ sudo modprobe uio
> +$ sudo insmod $RTE_SDK/x86_64-native-linuxapp-gcc/kmod/igb_uio.ko
> +$ lsmod | grep uio
> +igb_uio 16384 0 # igb_uio is loaded
> +uio 20480 1 igb_uio
> +```
> +
> +Then, bind it with PCI_Number.
> +```sh
> +$ $RTE_SDK/usertools/dpdk-devbind.py --status
> +# check your device for PCI_Number
> +
> +$ sudo $RTE_SDK/usertools/dpdk-devbind.py --bind=igb_uio [PCI_Number]
> +```
> +
> +#### virsh setup
> +
> +Edit VM configuration with virsh.
> +
> +```sh
> +$ virsh edit [VM_NAME]
> +<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
> + <name>spp-vm1</name>
> + <uuid>d90f5420-861a-4479-8559-62d7a1545cb9</uuid>
> + <memory unit='KiB'>4194304</memory>
> + <currentMemory unit='KiB'>4194304</currentMemory>
> + <memoryBacking>
> + <hugepages/>
> + </memoryBacking>
> + <vcpu placement='static'>4</vcpu>
> + <os>
> + <type arch='x86_64' machine='pc-i440fx-2.3'>hvm</type>
> + <boot dev='hd'/>
> + </os>
> + <features>
> + <acpi/>
> + <apic/>
> + <pae/>
> + </features>
> + <clock offset='utc'/>
> + <on_poweroff>destroy</on_poweroff>
> + <on_reboot>restart</on_reboot>
> + <on_crash>restart</on_crash>
> + <devices>
> + <emulator>/usr/local/bin/qemu-system-x86_64</emulator>
> + <disk type='file' device='disk'>
> + <driver name='qemu' type='raw'/>
> + <source file='/var/lib/libvirt/images/spp-vm1.qcow2'/>
> + <target dev='hda' bus='ide'/>
> + <address type='drive' controller='0' bus='0' target='0' unit='0'/>
> + </disk>
> + <disk type='block' device='cdrom'>
> + <driver name='qemu' type='raw'/>
> + <target dev='hdc' bus='ide'/>
> + <readonly/>
> + <address type='drive' controller='0' bus='1' target='0' unit='0'/>
> + </disk>
> + <controller type='usb' index='0'>
> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
> + </controller>
> + <controller type='pci' index='0' model='pci-root'/>
> + <controller type='ide' index='0'>
> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
> + </controller>
> + <interface type='network'>
> + <mac address='52:54:00:99:aa:7f'/>
> + <source network='default'/>
> + <model type='rtl8139'/>
> + <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
> + </interface>
> + <serial type='pty'>
> + <target type='isa-serial' port='0'/>
> + </serial>
> + <console type='pty'>
> + <target type='serial' port='0'/>
> + </console>
> + <memballoon model='virtio'>
> + <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
> + </memballoon>
> + </devices>
> + <qemu:commandline>
> + <qemu:arg value='-cpu'/>
> + <qemu:arg value='host'/>
> + <qemu:arg value='-object'/>
> + <qemu:arg value='memory-backend-file,id=mem,size=4096M,mem-path=/run/hugepages/kvm,share=on'/>
> + <qemu:arg value='-numa'/>
> + <qemu:arg value='node,memdev=mem'/>
> + <qemu:arg value='-mem-prealloc'/>
> + <qemu:arg value='-chardev'/>
> + <qemu:arg value='socket,id=chr0,path=/tmp/sock0'/>
> + <qemu:arg value='-device'/>
> + <qemu:arg value='virtio-net-pci,netdev=vhost-net0'/>
> + <qemu:arg value='-netdev'/>
> + <qemu:arg value='vhost-user,id=vhost-net0,chardev=chr0,vhostforce'/>
> + <qemu:arg value='-chardev'/>
> + <qemu:arg value='socket,id=chr1,path=/tmp/sock1'/>
> + <qemu:arg value='-device'/>
> + <qemu:arg value='virtio-net-pci,netdev=vhost-net1'/>
> + <qemu:arg value='-netdev'/>
> + <qemu:arg value='vhost-user,id=vhost-net1,chardev=chr1,vhostforce'/>
> + </qemu:commandline>
> + </domain>
> +```
> +
> +### Trouble Shooting
> +
> +You might encounter a permission error for `tmp/sockN` because of appamor.
> +In this case, you should try it.
> +
> +```sh
> +$ sudo ln -s /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper /etc/apparmor.d/disable/usr.lib.libvirt.virt-aa-helper
> +$ sudo ln -s /etc/apparmor.d/usr.sbin.libvirtd /etc/apparmor.d/disable/usr.sbin.libvirtd
> +$ sudo apparmor_parser -R /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper
> +$ sudo apparmor_parser -R /etc/apparmor.d/usr.sbin.libvirtd
> +$ sudo service apparmor reload
> +$ sudo service apparmor restart
> +$ sudo service libvirt-bin restart
> +```
> +
> +Or, you remove appamor.
> +
> +```sh
> +$ sudo apt-get remove apparmor
> +```
> +
> +If you use CentOS, not Ubuntu, confirm that SELinux doesn't prevent for permission.
> +SELinux should be disabled in this case.
> +
> +```sh
> +# /etc/selinux/config
> +SELINUX=disabled
> +```
> +
> +Check your SELinux configuration.
> +
> +```sh
> +$ getenforce
> +Disabled
> +```
>
--
Yasufumi Ogawa
NTT Network Service Systems Labs
More information about the spp
mailing list