[spp] [PATCH 11/30] doc: add sample usage
x-fn-spp at sl.ntt-tx.co.jp
x-fn-spp at sl.ntt-tx.co.jp
Tue Jan 16 06:16:22 CET 2018
From: Hiroyuki Nakamura <nakamura.hioryuki at po.ntt-tx.co.jp>
Signed-off-by: Naoki Takada <takada.naoki at lab.ntt.co.jp>
---
docs/spp_vf/how_to_use.md | 2 +-
docs/spp_vf/sample_usage.md | 212 ++++++++++++++++++++++++++++++++
docs/spp_vf/setup.md | 291 --------------------------------------------
docs/spp_vf/setup_guide.md | 291 ++++++++++++++++++++++++++++++++++++++++++++
docs/spp_vf/spp_vf.md | 6 +-
5 files changed, 509 insertions(+), 293 deletions(-)
create mode 100644 docs/spp_vf/sample_usage.md
delete mode 100644 docs/spp_vf/setup.md
create mode 100644 docs/spp_vf/setup_guide.md
diff --git a/docs/spp_vf/how_to_use.md b/docs/spp_vf/how_to_use.md
index ca9cf15..6b5019c 100644
--- a/docs/spp_vf/how_to_use.md
+++ b/docs/spp_vf/how_to_use.md
@@ -14,7 +14,7 @@ SPP_VF is a SR-IOV like network functionality for NFV.
## Launch SPP
-Before launching spp, you need to setup described as [setup guide](setup.md).
+Before launching spp, you need to setup described as [setup guide](setup_guide.md).
### SPP Controller
diff --git a/docs/spp_vf/sample_usage.md b/docs/spp_vf/sample_usage.md
new file mode 100644
index 0000000..2157378
--- /dev/null
+++ b/docs/spp_vf/sample_usage.md
@@ -0,0 +1,212 @@
+# SPP_VF Sample Usage
+
+This sample demonstrates an use-case of MAC address classification of
+spp_vf.
+It enables to access to VMs with ssh from a remote node.
+
+Before trying this sample, install SPP by following
+[setup guide](setup_guide.md).
+
+## Testing Steps
+
+In this section, you configure spp_vf and client applications for
+this use-case.
+
+
+### Setup SPP
+
+First, launch spp controller and primary process.
+
+ ```sh
+ $ pwd
+ /path/to/Soft-Patch-Panel
+
+ # SPP controller
+ $ python ./src/spp.py -p 5555 -s 6666
+
+ # SPP primary
+ $ sudo ./src/primary/x86_64-native-linuxapp-gcc/spp_primary \
+ -c 0x02 -n 4 --socket-mem 512,512 --huge-dir=/run/hugepages/kvm \
+ --proc-type=primary \
+ -- \
+ -p 0x03 -n 8 -s 127.0.0.1:5555
+ ```
+
+TODO(yasufum) add description for sec.
+
+ ```sh
+ # start nc for secondary 1
+ $ while true; do nc -l 11111; done
+
+ # start secondary 1
+ $ sudo ./src/vf/x86_64-native-linuxapp-gcc/spp_vf \
+ -c 0x00fd -n 4 --proc-type=secondary \
+ -- \
+ --process-id 1 \
+ --config $SPRINT_REVIEW_HOME/spp_vf1_without_cmtab.json \
+ -s 127.0.0.1:11111
+
+ # start nc for secondary 2
+ $ while true; do nc -l 11112; done
+
+ # start secondary 2
+ $ sudo ./src/vf/x86_64-native-linuxapp-gcc/spp_vf \
+ -c 0x3f01 -n 4 --proc-type=secondary \
+ -- \
+ --process-id 2 \
+ --config $SPRINT_REVIEW_HOME/spp_vf2_without_cmtab.json \
+ -s 127.0.0.1:11112
+ ```
+
+### Setup network configuration for VMs.
+
+Start two VMs.
+
+ ```sh
+ $ virsh start spp-vm1
+ $ virsh start spp-vm2
+ ```
+
+Login to spp-vm1 for network configuration.
+To not ask for unknown keys while login VMs,
+set `-oStrictHostKeyChecking=no` option for ssh.
+
+ ```sh
+ $ ssh -oStrictHostKeyChecking=no ntt at 192.168.122.31
+ ```
+
+Up interfaces for vhost and register them to arp table inside spp-vm1.
+In addition, you have to disable TCP offload function, or ssh is faled
+after configuration is done.
+
+ ```sh
+ # up interfaces
+ $ sudo ifconfig ens4 inet 192.168.240.21 netmask 255.255.255.0 up
+ $ sudo ifconfig ens5 inet 192.168.250.22 netmask 255.255.255.0 up
+
+ # register to arp table
+ $ sudo arp -s 192.168.240.11 a0:36:9f:78:86:78 -i ens4
+ $ sudo arp -s 192.168.250.13 a0:36:9f:6c:ed:bc -i ens5
+
+ # diable TCP offload
+ $ sudo ethtool -K ens4 tx off
+ $ sudo ethtool -K ens5 tx off
+ ```
+
+Configurations for spp-vm2 is same as spp-vm1.
+
+ ```sh
+ $ ssh -oStrictHostKeyChecking=no ntt at 192.168.122.32
+
+ # up interfaces
+ $ sudo ifconfig ens4 inet 192.168.240.31 netmask 255.255.255.0 up
+ $ sudo ifconfig ens5 inet 192.168.250.32 netmask 255.255.255.0 up
+
+ # register to arp table
+ $ sudo arp -s 192.168.240.11 a0:36:9f:78:86:78 -i ens4
+ $ sudo arp -s 192.168.250.13 a0:36:9f:6c:ed:bc -i ens5
+
+ # diable TCP offload
+ $ sudo ethtool -K ens4 tx off
+ $ sudo ethtool -K ens5 tx off
+ ```
+
+Check the configuration by trying ssh from remote machine that
+connection is accepted but discarded in spp secondary.
+If you do ssh for VM1, you find a messages from spp secondary for
+discarding packets.
+
+## Test Application
+
+TODO(yasufum) json-based steps are deprecated.
+
+### Register MAC address to Classifier
+
+Send a request for getting each of process IDs with nc command.
+TODO(yasufum) for what?
+
+ ```sh
+ {
+ "commands": [
+ {
+ "command": "process"
+ }
+ ]
+ }
+ ```
+
+Register MAC addresses to classifier.
+
+ ```sh
+ {
+ "commands": [
+ {
+ "command": "classifier_table",
+ "type": "mac",
+ "value": "52:54:00:12:34:56",
+ "port": "ring0"
+ },
+ {
+ "command": "classifier_table",
+ "type": "mac",
+ "value": "52:54:00:12:34:58",
+ "port": "ring1"
+ },
+ {
+ "command": "flush"
+ }
+ ]
+ }
+ ```
+
+ ```sh
+ {
+ "commands": [
+ {
+ "command": "classifier_table",
+ "type": "mac",
+ "value": "52:54:00:12:34:57",
+ "port": "ring4"
+ },
+ {
+ "command": "flush"
+ }
+ ]
+ }
+ ```
+
+### Login to VMs
+
+Now, you can login VMs.
+
+ ```sh
+ # spp-vm1 via NIC0
+ $ ssh ntt at 192.168.240.21
+
+ # spp-vm1 via NIC1
+ $ ssh ntt at 192.168.250.22
+
+ # spp-vm2 via NIC0
+ $ ssh ntt at 192.168.240.31
+
+ # spp-vm2 via NIC1
+ $ ssh ntt at 192.168.250.32
+ ```
+
+If you unregister the addresses, send request as following.
+
+ ```sh
+ {
+ "commands": [
+ {
+ "command": "classifier_table",
+ "type": "mac",
+ "value": "52:54:00:12:34:58",
+ "port": "unuse"
+ },
+ {
+ "command": "flush"
+ }
+ ]
+ }
+ ```
diff --git a/docs/spp_vf/setup.md b/docs/spp_vf/setup.md
deleted file mode 100644
index 450e68a..0000000
--- a/docs/spp_vf/setup.md
+++ /dev/null
@@ -1,291 +0,0 @@
-# Setup Guide
-
-## Environment
-
-* Ubuntu 16.04
-* qemu-kvm
-* DPDK v17.05
-
-## Setting
-
-### Host
-
-#### Edit Config
-
-Uncomment user and group in `/etc/libvirt/qemu.conf`.
-
-```sh
-# /etc/libvirt/qemu.conf
-
-user = "root"
-group = "root"
-```
-
-Change `KVM_HUGEPAGES` from 0 to 1 in `/etc/default/qemu-kvm`.
-
-```sh
-# /etc/default/qemu-kvm
-
-KVM_HUGEPAGES=1
-```
-
-Change grub config for hugepages and isolcpus.
-
-```sh
-# /etc/default/grub
-
-GRUB_CMDLINE_LINUX_DEFAULT="isolcpus=2,4,6,8,10,12-18,20,22,24,26-42,44,46 hugepagesz=1G hugepages=36 default_hugepagesz=1G"
-```
-
-You need to run `update-grub` and reboot to activate grub config.
-
-```sh
-$ sudo upadte-grub
-$ sudo reboot
-```
-
-You can check hugepage settings as following.
-
-```sh
-$ cat /proc/meminfo | grep -i huge
-AnonHugePages: 2048 kB
-HugePages_Total: 36 # /etc/default/grub
-HugePages_Free: 36
-HugePages_Rsvd: 0
-HugePages_Surp: 0
-Hugepagesize: 1048576 kB # /etc/default/grub
-
-$ mount | grep -i huge
-cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb,release_agent=/run/cgmanager/agents/cgm-release-agent.hugetlb,nsroot=/)
-hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
-hugetlbfs-kvm on /run/hugepages/kvm type hugetlbfs (rw,relatime,mode=775,gid=117)
-hugetlb on /run/lxcfs/controllers/hugetlb type cgroup (rw,relatime,hugetlb,release_agent=/run/cgmanager/agents/cgm-release-agent.hugetlb,nsroot=/)
-```
-
-Finally, you unmount default hugepage.
-
-```sh
-$ sudo unmount /dev/hugepages
-```
-
-#### Install jasson
-
-Network configuration is defined in JSON and `spp_vf` reads config from
-the file while launching.
-[jasson](http://www.digip.org/jansson/) is a JSON library written in C.
-
-It is required to use `json_path` feature of `jasson` for `spp_vf`.
-It has develped under `json_path` branch and you need to checkout and compile
-it manually.
-
-```sh
-$ git clone https://github.com/rogerz/jansson
-$ cd jansson
-$ sudo git checkout json_path
-Branch json_path set up to track remote branch json_path from origin. Switched to a new branch 'json_path'
-```
-
-This setup guide expects that `jasson` is placed as `/opt/jasson`.
-
-```sh
-$ sudo mkdir -p /opt/jansson
-$ sudo mv jansson /opt/jansson
-```
-
-Compile it as following.
-
-```sh
-$ cd /opt/jansson/jansson
-$ sudo autoreconf -i
-$ sudo ./configure
-$ sudo make
-$ sudo make install
-$ sudo ldconfig
-```
-
-Then, confirm that header files of jasson are generated in `/usr/local/include`.
-
-```sh
-$ ls -al /usr/local/include
-total 24
-drwxr-xr-x 2 root root 4096 Jul 28 16:45 .
-drwxr-xr-x 10 root root 4096 May 27 10:23 ..
--rw-r--r-- 1 root root 1183 Jul 28 16:45 jansson_config.h
--rw-r--r-- 1 root root 9499 Jul 28 16:45 jansson.h
-```
-
-#### Install DPDK
-
-Install DPDK v17.05 in any directory. This is a simple instruction and please refer
-[Getting Started Guide for Linux](http://dpdk.org/doc/guides/linux_gsg/index.html)
-for details.
-
-```sh
-$ cd /path/to/any_dir
-$ git clone http://dpdk.org/git/dpdk
-$ cd dpdk
-$ git checkout v17.05
-$ export RTE_SDK=`pwd`
-$ export RTE_TARGET=x86_64-native-linuxapp-gcc
-$ make T=x86_64-native-linuxapp-gcc install
-```
-
-#### Install SPP
-
-Clone SPP in any directory and compile it.
-
-```sh
-$ cd /path/to/spp_home/
-$ git clone https://github.com/ntt-ns/Soft-Patch-Panel.git
-export SPP_HOME=/path/to/spp_home/Soft-Patch-Panel
-$ cd $SPP_HOME
-$ make
-```
-
-#### Setup for DPDK
-
-Load igb_uio module.
-
-```sh
-$ sudo modprobe uio
-$ sudo insmod $RTE_SDK/x86_64-native-linuxapp-gcc/kmod/igb_uio.ko
-$ lsmod | grep uio
-igb_uio 16384 0 # igb_uio is loaded
-uio 20480 1 igb_uio
-```
-
-Then, bind it with PCI_Number.
-```sh
-$ $RTE_SDK/usertools/dpdk-devbind.py --status
-# check your device for PCI_Number
-
-$ sudo $RTE_SDK/usertools/dpdk-devbind.py --bind=igb_uio [PCI_Number]
-```
-
-#### virsh setup
-
-Edit VM configuration with virsh.
-
-```sh
-$ virsh edit [VM_NAME]
-```
-
-```xml
-<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
- <name>spp-vm1</name>
- <uuid>d90f5420-861a-4479-8559-62d7a1545cb9</uuid>
- <memory unit='KiB'>4194304</memory>
- <currentMemory unit='KiB'>4194304</currentMemory>
- <memoryBacking>
- <hugepages/>
- </memoryBacking>
- <vcpu placement='static'>4</vcpu>
- <os>
- <type arch='x86_64' machine='pc-i440fx-2.3'>hvm</type>
- <boot dev='hd'/>
- </os>
- <features>
- <acpi/>
- <apic/>
- <pae/>
- </features>
- <clock offset='utc'/>
- <on_poweroff>destroy</on_poweroff>
- <on_reboot>restart</on_reboot>
- <on_crash>restart</on_crash>
- <devices>
- <emulator>/usr/local/bin/qemu-system-x86_64</emulator>
- <disk type='file' device='disk'>
- <driver name='qemu' type='raw'/>
- <source file='/var/lib/libvirt/images/spp-vm1.qcow2'/>
- <target dev='hda' bus='ide'/>
- <address type='drive' controller='0' bus='0' target='0' unit='0'/>
- </disk>
- <disk type='block' device='cdrom'>
- <driver name='qemu' type='raw'/>
- <target dev='hdc' bus='ide'/>
- <readonly/>
- <address type='drive' controller='0' bus='1' target='0' unit='0'/>
- </disk>
- <controller type='usb' index='0'>
- <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
- </controller>
- <controller type='pci' index='0' model='pci-root'/>
- <controller type='ide' index='0'>
- <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
- </controller>
- <interface type='network'>
- <mac address='52:54:00:99:aa:7f'/>
- <source network='default'/>
- <model type='rtl8139'/>
- <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
- </interface>
- <serial type='pty'>
- <target type='isa-serial' port='0'/>
- </serial>
- <console type='pty'>
- <target type='serial' port='0'/>
- </console>
- <memballoon model='virtio'>
- <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
- </memballoon>
- </devices>
- <qemu:commandline>
- <qemu:arg value='-cpu'/>
- <qemu:arg value='host'/>
- <qemu:arg value='-object'/>
- <qemu:arg value='memory-backend-file,id=mem,size=4096M,mem-path=/run/hugepages/kvm,share=on'/>
- <qemu:arg value='-numa'/>
- <qemu:arg value='node,memdev=mem'/>
- <qemu:arg value='-mem-prealloc'/>
- <qemu:arg value='-chardev'/>
- <qemu:arg value='socket,id=chr0,path=/tmp/sock0'/>
- <qemu:arg value='-device'/>
- <qemu:arg value='virtio-net-pci,netdev=vhost-net0'/>
- <qemu:arg value='-netdev'/>
- <qemu:arg value='vhost-user,id=vhost-net0,chardev=chr0,vhostforce'/>
- <qemu:arg value='-chardev'/>
- <qemu:arg value='socket,id=chr1,path=/tmp/sock1'/>
- <qemu:arg value='-device'/>
- <qemu:arg value='virtio-net-pci,netdev=vhost-net1'/>
- <qemu:arg value='-netdev'/>
- <qemu:arg value='vhost-user,id=vhost-net1,chardev=chr1,vhostforce'/>
- </qemu:commandline>
- </domain>
-```
-
-### Trouble Shooting
-
-You might encounter a permission error for `tmp/sockN` because of appamor.
-In this case, you should try it.
-
-```sh
-$ sudo ln -s /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper /etc/apparmor.d/disable/usr.lib.libvirt.virt-aa-helper
-$ sudo ln -s /etc/apparmor.d/usr.sbin.libvirtd /etc/apparmor.d/disable/usr.sbin.libvirtd
-$ sudo apparmor_parser -R /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper
-$ sudo apparmor_parser -R /etc/apparmor.d/usr.sbin.libvirtd
-$ sudo service apparmor reload
-$ sudo service apparmor restart
-$ sudo service libvirt-bin restart
-```
-
-Or, you remove appamor.
-
-```sh
-$ sudo apt-get remove apparmor
-```
-
-If you use CentOS, not Ubuntu, confirm that SELinux doesn't prevent for permission.
-SELinux should be disabled in this case.
-
-```sh
-# /etc/selinux/config
-SELINUX=disabled
-```
-
-Check your SELinux configuration.
-
-```sh
-$ getenforce
-Disabled
-```
diff --git a/docs/spp_vf/setup_guide.md b/docs/spp_vf/setup_guide.md
new file mode 100644
index 0000000..450e68a
--- /dev/null
+++ b/docs/spp_vf/setup_guide.md
@@ -0,0 +1,291 @@
+# Setup Guide
+
+## Environment
+
+* Ubuntu 16.04
+* qemu-kvm
+* DPDK v17.05
+
+## Setting
+
+### Host
+
+#### Edit Config
+
+Uncomment user and group in `/etc/libvirt/qemu.conf`.
+
+```sh
+# /etc/libvirt/qemu.conf
+
+user = "root"
+group = "root"
+```
+
+Change `KVM_HUGEPAGES` from 0 to 1 in `/etc/default/qemu-kvm`.
+
+```sh
+# /etc/default/qemu-kvm
+
+KVM_HUGEPAGES=1
+```
+
+Change grub config for hugepages and isolcpus.
+
+```sh
+# /etc/default/grub
+
+GRUB_CMDLINE_LINUX_DEFAULT="isolcpus=2,4,6,8,10,12-18,20,22,24,26-42,44,46 hugepagesz=1G hugepages=36 default_hugepagesz=1G"
+```
+
+You need to run `update-grub` and reboot to activate grub config.
+
+```sh
+$ sudo upadte-grub
+$ sudo reboot
+```
+
+You can check hugepage settings as following.
+
+```sh
+$ cat /proc/meminfo | grep -i huge
+AnonHugePages: 2048 kB
+HugePages_Total: 36 # /etc/default/grub
+HugePages_Free: 36
+HugePages_Rsvd: 0
+HugePages_Surp: 0
+Hugepagesize: 1048576 kB # /etc/default/grub
+
+$ mount | grep -i huge
+cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb,release_agent=/run/cgmanager/agents/cgm-release-agent.hugetlb,nsroot=/)
+hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
+hugetlbfs-kvm on /run/hugepages/kvm type hugetlbfs (rw,relatime,mode=775,gid=117)
+hugetlb on /run/lxcfs/controllers/hugetlb type cgroup (rw,relatime,hugetlb,release_agent=/run/cgmanager/agents/cgm-release-agent.hugetlb,nsroot=/)
+```
+
+Finally, you unmount default hugepage.
+
+```sh
+$ sudo unmount /dev/hugepages
+```
+
+#### Install jasson
+
+Network configuration is defined in JSON and `spp_vf` reads config from
+the file while launching.
+[jasson](http://www.digip.org/jansson/) is a JSON library written in C.
+
+It is required to use `json_path` feature of `jasson` for `spp_vf`.
+It has develped under `json_path` branch and you need to checkout and compile
+it manually.
+
+```sh
+$ git clone https://github.com/rogerz/jansson
+$ cd jansson
+$ sudo git checkout json_path
+Branch json_path set up to track remote branch json_path from origin. Switched to a new branch 'json_path'
+```
+
+This setup guide expects that `jasson` is placed as `/opt/jasson`.
+
+```sh
+$ sudo mkdir -p /opt/jansson
+$ sudo mv jansson /opt/jansson
+```
+
+Compile it as following.
+
+```sh
+$ cd /opt/jansson/jansson
+$ sudo autoreconf -i
+$ sudo ./configure
+$ sudo make
+$ sudo make install
+$ sudo ldconfig
+```
+
+Then, confirm that header files of jasson are generated in `/usr/local/include`.
+
+```sh
+$ ls -al /usr/local/include
+total 24
+drwxr-xr-x 2 root root 4096 Jul 28 16:45 .
+drwxr-xr-x 10 root root 4096 May 27 10:23 ..
+-rw-r--r-- 1 root root 1183 Jul 28 16:45 jansson_config.h
+-rw-r--r-- 1 root root 9499 Jul 28 16:45 jansson.h
+```
+
+#### Install DPDK
+
+Install DPDK v17.05 in any directory. This is a simple instruction and please refer
+[Getting Started Guide for Linux](http://dpdk.org/doc/guides/linux_gsg/index.html)
+for details.
+
+```sh
+$ cd /path/to/any_dir
+$ git clone http://dpdk.org/git/dpdk
+$ cd dpdk
+$ git checkout v17.05
+$ export RTE_SDK=`pwd`
+$ export RTE_TARGET=x86_64-native-linuxapp-gcc
+$ make T=x86_64-native-linuxapp-gcc install
+```
+
+#### Install SPP
+
+Clone SPP in any directory and compile it.
+
+```sh
+$ cd /path/to/spp_home/
+$ git clone https://github.com/ntt-ns/Soft-Patch-Panel.git
+export SPP_HOME=/path/to/spp_home/Soft-Patch-Panel
+$ cd $SPP_HOME
+$ make
+```
+
+#### Setup for DPDK
+
+Load igb_uio module.
+
+```sh
+$ sudo modprobe uio
+$ sudo insmod $RTE_SDK/x86_64-native-linuxapp-gcc/kmod/igb_uio.ko
+$ lsmod | grep uio
+igb_uio 16384 0 # igb_uio is loaded
+uio 20480 1 igb_uio
+```
+
+Then, bind it with PCI_Number.
+```sh
+$ $RTE_SDK/usertools/dpdk-devbind.py --status
+# check your device for PCI_Number
+
+$ sudo $RTE_SDK/usertools/dpdk-devbind.py --bind=igb_uio [PCI_Number]
+```
+
+#### virsh setup
+
+Edit VM configuration with virsh.
+
+```sh
+$ virsh edit [VM_NAME]
+```
+
+```xml
+<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
+ <name>spp-vm1</name>
+ <uuid>d90f5420-861a-4479-8559-62d7a1545cb9</uuid>
+ <memory unit='KiB'>4194304</memory>
+ <currentMemory unit='KiB'>4194304</currentMemory>
+ <memoryBacking>
+ <hugepages/>
+ </memoryBacking>
+ <vcpu placement='static'>4</vcpu>
+ <os>
+ <type arch='x86_64' machine='pc-i440fx-2.3'>hvm</type>
+ <boot dev='hd'/>
+ </os>
+ <features>
+ <acpi/>
+ <apic/>
+ <pae/>
+ </features>
+ <clock offset='utc'/>
+ <on_poweroff>destroy</on_poweroff>
+ <on_reboot>restart</on_reboot>
+ <on_crash>restart</on_crash>
+ <devices>
+ <emulator>/usr/local/bin/qemu-system-x86_64</emulator>
+ <disk type='file' device='disk'>
+ <driver name='qemu' type='raw'/>
+ <source file='/var/lib/libvirt/images/spp-vm1.qcow2'/>
+ <target dev='hda' bus='ide'/>
+ <address type='drive' controller='0' bus='0' target='0' unit='0'/>
+ </disk>
+ <disk type='block' device='cdrom'>
+ <driver name='qemu' type='raw'/>
+ <target dev='hdc' bus='ide'/>
+ <readonly/>
+ <address type='drive' controller='0' bus='1' target='0' unit='0'/>
+ </disk>
+ <controller type='usb' index='0'>
+ <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
+ </controller>
+ <controller type='pci' index='0' model='pci-root'/>
+ <controller type='ide' index='0'>
+ <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
+ </controller>
+ <interface type='network'>
+ <mac address='52:54:00:99:aa:7f'/>
+ <source network='default'/>
+ <model type='rtl8139'/>
+ <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
+ </interface>
+ <serial type='pty'>
+ <target type='isa-serial' port='0'/>
+ </serial>
+ <console type='pty'>
+ <target type='serial' port='0'/>
+ </console>
+ <memballoon model='virtio'>
+ <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
+ </memballoon>
+ </devices>
+ <qemu:commandline>
+ <qemu:arg value='-cpu'/>
+ <qemu:arg value='host'/>
+ <qemu:arg value='-object'/>
+ <qemu:arg value='memory-backend-file,id=mem,size=4096M,mem-path=/run/hugepages/kvm,share=on'/>
+ <qemu:arg value='-numa'/>
+ <qemu:arg value='node,memdev=mem'/>
+ <qemu:arg value='-mem-prealloc'/>
+ <qemu:arg value='-chardev'/>
+ <qemu:arg value='socket,id=chr0,path=/tmp/sock0'/>
+ <qemu:arg value='-device'/>
+ <qemu:arg value='virtio-net-pci,netdev=vhost-net0'/>
+ <qemu:arg value='-netdev'/>
+ <qemu:arg value='vhost-user,id=vhost-net0,chardev=chr0,vhostforce'/>
+ <qemu:arg value='-chardev'/>
+ <qemu:arg value='socket,id=chr1,path=/tmp/sock1'/>
+ <qemu:arg value='-device'/>
+ <qemu:arg value='virtio-net-pci,netdev=vhost-net1'/>
+ <qemu:arg value='-netdev'/>
+ <qemu:arg value='vhost-user,id=vhost-net1,chardev=chr1,vhostforce'/>
+ </qemu:commandline>
+ </domain>
+```
+
+### Trouble Shooting
+
+You might encounter a permission error for `tmp/sockN` because of appamor.
+In this case, you should try it.
+
+```sh
+$ sudo ln -s /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper /etc/apparmor.d/disable/usr.lib.libvirt.virt-aa-helper
+$ sudo ln -s /etc/apparmor.d/usr.sbin.libvirtd /etc/apparmor.d/disable/usr.sbin.libvirtd
+$ sudo apparmor_parser -R /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper
+$ sudo apparmor_parser -R /etc/apparmor.d/usr.sbin.libvirtd
+$ sudo service apparmor reload
+$ sudo service apparmor restart
+$ sudo service libvirt-bin restart
+```
+
+Or, you remove appamor.
+
+```sh
+$ sudo apt-get remove apparmor
+```
+
+If you use CentOS, not Ubuntu, confirm that SELinux doesn't prevent for permission.
+SELinux should be disabled in this case.
+
+```sh
+# /etc/selinux/config
+SELINUX=disabled
+```
+
+Check your SELinux configuration.
+
+```sh
+$ getenforce
+Disabled
+```
diff --git a/docs/spp_vf/spp_vf.md b/docs/spp_vf/spp_vf.md
index 786e712..99c914f 100644
--- a/docs/spp_vf/spp_vf.md
+++ b/docs/spp_vf/spp_vf.md
@@ -29,12 +29,16 @@ by looking MAC address in the packet..
## Build the Application
-See [setup guide](setup.md).
+See [setup guide](setup_guide.md).
## Running the Application
See [how to use](how_to_use.md).
+## Sample Usage
+
+See [sample usage](sample_usage.md).
+
## Explanation
The following sections provide some explanation of the code.
--
1.9.1
More information about the spp
mailing list