Skip to content

Latest commit

 

History

History
146 lines (123 loc) · 5.45 KB

fig8c.md

File metadata and controls

146 lines (123 loc) · 5.45 KB

Figure 8(c) PCP: Physical port to Container to Physical port

This page describes how to setup packet forwarding between physical port and container port. Similiarly, we have three configurations, kernel, AF_XDP and DPDK.

Kernel Datapath

First, we start OVS using kernel datapath by running

/usr/share/openvswitch/scripts/ovs-ctl start

In addition, we create a namespace with veth device pair, p0 and ovs-p0, attached to the namespace, called at_ns0:

ip link add p0 type veth peer name ovs-p0
ip link set p0 netns at_ns0
ip link set dev ovs-p0 up

ip netns exec at_ns0 sh << NS_EXEC_HEREDOC
ip addr add "10.1.1.1/24" dev p0
ip link set dev p0 up
NS_EXEC_HEREDOC

This simulates the container networking environment using namespace and veth. Next, attach the port to OVS br0 and program the OpenFlow rules:

ovs-vsctl add-br br0
# add physical port
ovs-vsctl add-port br0 enp2s0f0
ovs-vsctl add-port br0 enp2s0f1

# add container veth port
ovs-vsctl add-port br0 ovs-p0

ovs-ofctl del-flows br0
# add PCP forwarding rule
ovs-ofctl add-flow br0 "in_port=enp2s0f0, actions=output:ovs-p0"
ovs-ofctl add-flow br0 "in_port=ovs-p0, actions=output:enp2s0f1"

At the container namespace, we have to setup the loopback forwarding rule to send back the packets. Again, we can use the xdp_rxq_info tool provided by Linux kernel.

ip link set ovs-p0 xdpdrv obj /root/xdp_noop.o section xdp
ip netns exec at_ns0 /root/bpf-next/samples/bpf/xdp_rxq_info --action XDP_TX -d p0

The xdp_rxq_info tool is part of the Linux kernel source. You can download and compile it by

git clone git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git
cd bpf-next/samples/bpf/
make

Finally, start the TRex traffic gen and observe the performance.

AF_XDP (Using OVS XDP offload feature)

To test OVS with AF_XDP using a XDP program loaded in the kernel, as explained in Figure 4, path (C), since the feature is not upstreamed to OVS yet, we use the implementation from NetDev0x14 with OVS XDP source code at link. Instructions about how to compile and setup XDP for OVS is here

Once you've compiled the OVS with XDP offload, start OVS with userspace datapath:

ovsdb-tool create /usr/local/etc/openvswitch/conf.db /root/ovs/vswitchd/vswitch.ovsschema
ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
    --remote=db:Open_vSwitch,Open_vSwitch,manager_options \
    --pidfile --detach

ovs-vsctl --no-wait init
ovs-vswitchd --no-chdir --pidfile --log-file=/root/ovs/ovs-vswitchd.log  --disable-system --detach
ovs-vsctl -- add-br br0 -- set Bridge br0 protocols=OpenFlow10,OpenFlow11,OpenFlow12,OpenFlow13,OpenFlow14,OpenFlow15 fail-mode=secure datapath_type=netdev

Next, create namespace with veth pair p0 and afxdp-p0.

# create namespace
ip netns add at_ns0
ip link add p0 type veth peer name afxdp-p0
ip link set p0 netns at_ns0
ip link set dev afxdp-p0 up
ovs-vsctl add-port br0 afxdp-p0

Assume you have ovs source code at /root/ovs. The folder ovs/bpf/ will contain a BPF bytecode, named flowtable_afxdp.o, which implemented the XDP offload. Now, attach the device to OVS bridge and load the XDP offload bytecode:

ovs-vsctl add-port br0 enp2s0f0 -- set int enp2s0f0 type=afxdp \
   options:n_rxq=1 options:xdp-mode=native options:xdp-obj="/root/ovs/bpf/flowtable_afxdp.o"
ovs-vsctl add-port br0 enp2s0f1 -- set int enp2s0f1 type=afxdp \
   options:n_rxq=1 options:xdp-mode=native options:xdp-obj="/root/ovs/bpf/flowtable_afxdp.o"
ovs-vsctl -- set interface afxdp-p0 type=afxdp options:xdp-mode=native \
    options:xdp-obj="/root/ovs/bpf/flowtable_afxdp.o"

#enable XDP offload
ovs-vsctl set Open_vSwitch . other_config:offload-driver=linux_xdp
ovs-vsctl set Open_vSwitch . other_config:hw-offload=true

Finally, follow the steps above to create loopback packets in the namespace, start the TRex, and measure the performance.

DPDK

Simiarly, create the namespace and veth devices, p0 and ovs-p0. Then start OVS-DPDK. You can build your own OVS-DPDK by following the OVS official document here.

Once OVS-DPDK is installed, follow the similar steps as above:

ovsdb-tool create /usr/local/etc/openvswitch/conf.db /src/ovs/vswitchd/vswitch.ovsschema
ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
    --remote=db:Open_vSwitch,Open_vSwitch,manager_options --pidfile --detach
ovs-vsctl --no-wai -- init
sleep 1

ovs-vswitchd --no-chdir --pidfile --log-file --disable-system --detach
ovs-vsctl show

# start OVS userspace datapath, "datapath_type=netdev"
ovs-vsctl add-br br0 -- set Bridge br0 protocols=OpenFlow10,OpenFlow11,OpenFlow12,OpenFlow13,OpenFlow14,OpenFlow15 fail-mode=secure datapath_type=netdev

ovs-vsctl add-port br0 enp2s0f0 -- set int enp2s0f0 type=dpdk \
    options:dpdk-devargs=0000:02:00.0
ovs-vsctl add-port br0 enp2s0f1 -- set int enp2s0f1 type=dpdk \
    options:dpdk-devargs=0000:02:00.1

# Add container veth
ovs-vsctl add-port br0 ovs-p0

# Add PCP forwarding rule
ovs-ofctl del-flows br0
ovs-ofctl add-flow br0 "in_port=enp2s0f0, actions=output:ovs-p0"
ovs-ofctl add-flow br0 "in_port=ovs-p0, actions=output:enp2s0f1"

Then, follow the similar steps above to setup loopback in the container namespace, start TRex, and measure the performance.