{"id":130,"date":"2013-09-20T20:01:07","date_gmt":"2013-09-20T18:01:07","guid":{"rendered":"http:\/\/www.opencloudblog.com\/?p=130"},"modified":"2013-09-23T20:14:34","modified_gmt":"2013-09-23T18:14:34","slug":"openvswitch-port-usage-on-linux","status":"publish","type":"post","link":"https:\/\/www.opencloudblog.com\/?p=130","title":{"rendered":"Openvswitch &#8211; Port Usage on Linux"},"content":{"rendered":"<p>In this post I show the basic commands for the openvswitch.\u00a0You should have at least openvswitch 1.9 installed. This is the case, if you are running Ubuntu 13.04.<\/p>\n<h1>Preparation<\/h1>\n<p>In this blog, I show more than the basic commands. We use Linux network namespaces to show also the usage on the ip layer. The first step is to create two linux network namespaces (you must be root):<\/p>\n<pre class=\"lang:sh decode:true\"># create two network namespaces\r\nip netns add ns1\r\nip netns add ns2\r\n\r\n# bring up the loopback interfaces of the namespaces\r\nip netns exec ns1 ip link set dev lo up\r\nip netns exec ns2 ip link set dev lo up<\/pre>\n<p>Next, \u00a0we need three console sessions. One for each network namespace (use ip netns exec ns &lt;x&gt; bash to get into the namespace) and one for the non namespace config. I recommend, that you change the prompt for each session &#8211; it \u00a0helps a lot, that you always knows, where you are.<\/p>\n<pre class=\"lang:sh decode:true\" title=\"Set Prompt for each console window\">#\r\n# MAIN bash \r\n#\r\nexport PS1=\"Main&gt;\"\r\n#\r\n# Console for Network Namespace ns1\r\n#\r\nip netns exec ns1 bash\r\nexport PS1=\"NS 1&gt;\"\r\n#\r\n# Console in Network Namespace ns2\r\n#\r\nip netns exec ns2 bash\r\nexport PS1=\"NS 2&gt;\"<\/pre>\n<h1>\u00a0Create a Switch (bridge)<\/h1>\n<p>The next step is to create a new virtual switch (or bridge). Use the following command in the Main console:<\/p>\n<pre class=\"lang:sh decode:true\" title=\"Create a switch\"># create a new virtual switch\r\novs-vsctl add-br testsw1\r\n\r\n# check using\r\novs-vsctl show\r\n\r\n# the output will look like\r\n9ddae96d-0cc4-4c17-98c2-7ffe3f4f6226             &lt;==== this is the UUID of the running openvswitch\r\n    Bridge testsw1                               &lt;==== this is a running virtual switch instance\r\n        Port testsw1                             \r\n            Interface testsw1                    &lt;==== By default, each virtual switch gets an internal interface      \r\n                type: internal                         internal interface == this is a interface seen by Linux\r\n    ovs_version: \"1.9.0\"                         &lt;==== the version of the openvswitch software running<\/pre>\n<p>By default, each virtual switch (=bridge), which has been created, gets one <em>&#8222;internal port&#8220;<\/em> assigned. These internal ports are seen by the Linux OS. Type &#8222;ip link&#8220; in the Main console and you get:<\/p>\n<pre class=\"lang:sh decode:true\" title=\"The openvswitch bridge is seen by the host OS\">19: testsw1: &lt;BROADCAST,MULTICAST&gt; mtu 1500 qdisc noop state DOWN mode DEFAULT \r\n    link\/ether ae:fd:bb:27:2b:43 brd ff:ff:ff:ff:ff:ff<\/pre>\n<p>You can use this interface in the same way as all other Linux interfaces. Assign IP addresses to it or put IP access lists on it (iptables). We will ignore this interface.<\/p>\n<p>An openvswitch bridge is not only a simple bridge, it&#8217;s a full blown software switch with<\/p>\n<ul>\n<li>4 k Vlans &#8211; nothing to do to create them &#8211; they are just there<\/li>\n<li>it&#8217;s own MAC table<\/li>\n<\/ul>\n<h1>Create two ports in the bridge<\/h1>\n<p>The next step is to create two ports in the switch testsw1 and assign these to the two network namespaces. Run this in the Main console.<\/p>\n<pre class=\"lang:sh decode:true\"># create an internal ovs port\r\novs-vsctl add-port testsw1 ns1iface1 -- set Interface ns1iface1 type=internal\r\n# attach it to namespace\r\nip link set ns1iface1 netns ns1\r\n\r\n# create an internal ovs port\r\novs-vsctl add-port testsw1 ns2iface1 -- set Interface ns2iface1 type=internal\r\n# attach it to namespace\r\nip link set ns2iface1 netns ns2<\/pre>\n<p>Check now in the Main console the openvswitch:<\/p>\n<pre class=\"lang:sh decode:true\" title=\"2 ports have been created\"># show all switches\r\novs-vsctl show\r\n\r\n    Bridge \"testsw1\"\r\n        Port \"ns2iface1\"\r\n            Interface \"ns2iface1\"\r\n                type: internal\r\n        Port \"testsw1\"\r\n            Interface \"testsw1\"\r\n                type: internal\r\n        Port \"ns1iface1\"\r\n            Interface \"ns1iface1\"\r\n                type: internal<\/pre>\n<p>We see the two ports in the list. A shorter command to list only the ports is<\/p>\n<pre class=\"lang:sh decode:true\" title=\"List all ports\"># list all ports of a bridge - here testsw1\r\novs-vsctl list-ports testsw1\r\nns1iface1\r\nns2iface1<\/pre>\n<p>Now look into the namespace windows. Type <strong>ip link<\/strong> in both windows and the new interfaces appeared.<\/p>\n<p>In ns1:<\/p>\n<pre class=\"lang:sh decode:true\" title=\"in ns1\">ip l\r\n1: lo: &lt;LOOPBACK,UP,LOWER_UP&gt; mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT \r\n    link\/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00\r\n20: ns1iface1: &lt;BROADCAST,MULTICAST&gt; mtu 1500 qdisc noop state DOWN mode DEFAULT \r\n    link\/ether 5e:e7:51:a5:10:55 brd ff:ff:ff:ff:ff:ff\r\n#\r\n# bring up ns1iface1\r\n#\r\nip link set dev ns1iface1 up\r\n# \r\n# assign an ip address\r\n#\r\nip addr add dev ns1iface1 10.0.0.1\/24<\/pre>\n<p>In ns2:<\/p>\n<pre class=\"lang:sh decode:true\" title=\"in ns2\">ip l\r\n1: lo: &lt;LOOPBACK,UP,LOWER_UP&gt; mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT \r\n    link\/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00\r\n21: ns2iface1: &lt;BROADCAST,MULTICAST&gt; mtu 1500 qdisc noop state DOWN mode DEFAULT \r\n    link\/ether 8a:42:b3:91:db:c2 brd ff:ff:ff:ff:ff:ff\r\n#\r\n# bring up ns2iface1\r\n#\r\nip link set dev ns2iface1 up\r\n# \r\n# assign an ip address\r\n#\r\nip addr add dev ns2iface1 10.0.0.2\/24<\/pre>\n<p>Now it should be possible to ping the other side.<\/p>\n<p>In the Main console check the MAC address table of testsw1:<\/p>\n<pre class=\"lang:sh decode:true\" title=\"List all MAC addresses of a switch\">#\r\n# list all mac addresses of testsw1 and the ports where the MAC addresses are located\r\n#\r\novs-appctl fdb\/show testsw1\r\n port  VLAN  MAC                Age\r\n    1     0  5e:e7:51:a5:10:55   58\r\n    2     0  8a:42:b3:91:db:c2   58<\/pre>\n<p>The MAC addresses match those of the interfaces, we previously created. You might notice, that both MAC addresses are attached to Vlan 0. So &#8211; what is Vlan 0?<\/p>\n<p><strong><em>By default, ALL ports\/interfaces, which are created on the openvswitch are able to transport dot1q tagged traffic unless there is a restriction added when the port is created.<\/em><\/strong> We made no restriction about this when we created the ports. Since both ip interfaces used in the network namespaces do not send dot1q tagged traffic, the openvswitch places this traffic in Vlan 0.<\/p>\n<p>This Vlan 0 means: the openvswitch got untagged traffic on a link, where dot1q transport is enabled.<\/p>\n<h1>Create a dot1q tagged link<\/h1>\n<p>Now we create a dot1q tagged link in both network namespaces on top of the existing link. the command to do this is <strong>ip<\/strong>.<\/p>\n<p>Example for network namespace ns1.<\/p>\n<pre class=\"lang:sh decode:true\" title=\"Add a tagged interface\">#\r\n# create a Vlan subinterface on a interface\r\n#\r\n# here (example with the interface in namespace ns1)\r\n# Vlan tag: 22\r\n#\r\nip link add link ns1iface1 name ns1iface1.22 type vlan id 22\r\n#\r\n# it's a good practice to code the vlan number and the interface in the interface name for the new interface\r\n#\r\n# check the existing interfaces using ip link\r\n#\r\nNS 1&gt;ip l\r\n1: lo: &lt;LOOPBACK,UP,LOWER_UP&gt; mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT \r\n    link\/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00\r\n2: ns1iface1.22@ns1iface1: &lt;BROADCAST,MULTICAST&gt; mtu 1500 qdisc noop state DOWN mode DEFAULT \r\n    link\/ether 5e:e7:51:a5:10:55 brd ff:ff:ff:ff:ff:ff\r\n20: ns1iface1: &lt;BROADCAST,MULTICAST,UP,LOWER_UP&gt; mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT \r\n    link\/ether 5e:e7:51:a5:10:55 brd ff:ff:ff:ff:ff:ff<\/pre>\n<p>Do the same for network namespace ns2 (do not forget to change the interface name!)<\/p>\n<p>Then bring the interface up and assign an ip address.<\/p>\n<p>ns1 and ns2:<\/p>\n<pre class=\"lang:sh decode:true\" title=\"for ns1 and ns2\">#\r\n#### N S 1 ######\r\n#\r\n# bring the interface up\r\nip link set dev ns1iface1.22 up\r\n#\r\n# assign an ip address\r\nip addr add dev ns1iface1.22 10.1.0.1\/24\r\n#\r\n#### N S 2 ######\r\n#\r\n# bring the interface up\r\nip link set dev ns1iface1.22 up\r\n#\r\n# assign an ip address\r\nip addr add dev ns1iface1.22 10.1.0.1\/24\r\n#\r\n#### N S 2 #####\r\n#\r\n# ping the other side (from ns2 to ns1) in vlan 22 \r\nping 10.1.0.1 \r\n#\r\n#### N S 1 ######\r\n#\r\n# ping the other side (from ns1 to ns2) on the untagged link \r\nping 10.1.0.2<\/pre>\n<p>Look now on the MAC table of the ovenvswitch in the Main window:<\/p>\n<pre class=\"lang:sh decode:true\" title=\"MAC Table\">ovs-appctl fdb\/show testsw1\r\n port  VLAN  MAC                Age\r\n    1    22  5e:e7:51:a5:10:55   87\r\n    2    22  8a:42:b3:91:db:c2   87\r\n    2     0  8a:42:b3:91:db:c2    1\r\n    1     0  5e:e7:51:a5:10:55    1<\/pre>\n<p>You see now two Vlans with active MAC addresses. Yes they are the same in both Vlans. This is OK, since each Vlan has it&#8217;s own layer 2 forwarding table.<\/p>\n<p>If the MAC entries disappear &#8211; don&#8217;t panic. This is a normal behaviour. As soon as traffic flows, they are populated again.<\/p>\n<p>We have now the following setup:<\/p>\n<div id=\"attachment_153\" style=\"width: 635px\" class=\"wp-caption aligncenter\"><a href=\"http:\/\/www.opencloudblog.com\/wp-content\/uploads\/2013\/09\/ovs-iface1.png\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-153\" class=\"size-full wp-image-153 \" alt=\"After the first port has been added\" src=\"http:\/\/www.opencloudblog.com\/wp-content\/uploads\/2013\/09\/ovs-iface1.png\" width=\"625\" height=\"227\" srcset=\"https:\/\/www.opencloudblog.com\/wp-content\/uploads\/2013\/09\/ovs-iface1.png 625w, https:\/\/www.opencloudblog.com\/wp-content\/uploads\/2013\/09\/ovs-iface1-150x54.png 150w\" sizes=\"(max-width: 625px) 100vw, 625px\" \/><\/a><p id=\"caption-attachment-153\" class=\"wp-caption-text\">By default a port created without restrictions will transport all vlan tags<\/p><\/div>\n<h1>Create a untagged port<\/h1>\n<p>The next step is to create ports on the openvswitch for the network namespaces, which are untagged by default. This is the case in 99.9% of all cases, when you are attaching LXC or KVM instances to an openvswitch. We attach these ports to vlan 33. This is done using the commands in the Main console:<\/p>\n<pre class=\"lang:sh decode:true\" title=\"untagged openvswitch ports\"># create an internal ovs port\r\novs-vsctl add-port testsw1 ns1iface2 tag=33 -- set Interface ns1iface2 type=internal\r\n# attach it to namespace\r\nip link set ns1iface2 netns ns1\r\n\r\n# create an internal ovs port\r\novs-vsctl add-port testsw1 ns2iface2 tag=33 -- set Interface ns2iface2 type=internal\r\n# attach it to namespace\r\nip link set ns2iface2 netns ns2<\/pre>\n<p>By adding &#8222;tag=33&#8220; while using the add-port the port is by default untagged (no dot1q tags are accepted) and the port is attached to Vlan 33 on the openvswitch.<\/p>\n<p>A ovs-vsctl show executed in the Main console outputs now:<\/p>\n<pre class=\"lang:sh decode:true\" title=\"ovs-vsctl show\">    Bridge \"testsw1\"\r\n        Port \"ns2iface1\"\r\n            Interface \"ns2iface1\"\r\n                type: internal\r\n        Port \"ns1iface2\"\r\n            tag: 33\r\n            Interface \"ns1iface2\"\r\n                type: internal\r\n        Port \"testsw1\"\r\n            Interface \"testsw1\"\r\n                type: internal\r\n        Port \"ns2iface2\"\r\n            tag: 33\r\n            Interface \"ns2iface2\"\r\n                type: internal\r\n        Port \"ns1iface1\"\r\n            Interface \"ns1iface1\"\r\n                type: internal<\/pre>\n<p>The ports ns1iface2 and ns2iface2 have now one additional line: &#8222;tag :33&#8220;. This means. The port is untagged and attached to Vlan 33. It does not mean, that only the Vlan tag 33 is accepted on a tagged link.<\/p>\n<p>Next task: Bring the new interfaces up in the network namespaces, assign the ip addresses 10.2.0.1 for ns1 and 10.2.0.2 for ns2 and try to ping. Look at the MAC table on the openvswitch (from the Main console). You should see MAC addresses in Vlan 33.<\/p>\n<p>We have now the following setup:<\/p>\n<div id=\"attachment_154\" style=\"width: 635px\" class=\"wp-caption aligncenter\"><a href=\"http:\/\/www.opencloudblog.com\/wp-content\/uploads\/2013\/09\/ovs-iface2.png\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-154\" class=\"size-full wp-image-154 \" alt=\"Add an untagged port\" src=\"http:\/\/www.opencloudblog.com\/wp-content\/uploads\/2013\/09\/ovs-iface2.png\" width=\"625\" height=\"227\" srcset=\"https:\/\/www.opencloudblog.com\/wp-content\/uploads\/2013\/09\/ovs-iface2.png 625w, https:\/\/www.opencloudblog.com\/wp-content\/uploads\/2013\/09\/ovs-iface2-150x54.png 150w\" sizes=\"(max-width: 625px) 100vw, 625px\" \/><\/a><p id=\"caption-attachment-154\" class=\"wp-caption-text\">A port with tag= will be untagged on the link and will be attached to a fixed vlan on the openvswitch<\/p><\/div>\n<h1>Create a dot1q trunking port with a restricted vlan list<\/h1>\n<p>The next step is to create a new interface, which is dot1q tagged, but allows only a limited list of tags (=vlans) on the port.<\/p>\n<p>The commands for the Main console are:<\/p>\n<pre class=\"lang:sh decode:true\" title=\"Create a trunk port\"># create an internal ovs port\r\novs-vsctl add-port testsw1 ns1iface3 trunk=44,55,66 -- set Interface ns1iface3 type=internal\r\n# attach it to namespace\r\nip link set ns1iface3 netns ns1\r\n\r\n# create an internal ovs port\r\novs-vsctl add-port testsw1 ns2iface3 trunk=44,55,66 -- set Interface ns2iface3 type=internal\r\n# attach it to namespace\r\nip link set ns2iface3 netns ns2<\/pre>\n<p>The keyword trunk tell the ovs to allow dot1q on this port. We allow the tags (=vlans) 44, 55 and 66.<\/p>\n<p>The command &#8222;ovs-vsctl show&#8220; shows now:<\/p>\n<pre class=\"lang:sh decode:true\" title=\"ovs-vsctl show\">    Bridge \"testsw1\"\r\n        Port \"ns1iface3\"\r\n            trunks: [44, 55, 66]\r\n            Interface \"ns1iface3\"\r\n                type: internal\r\n        Port \"ns2iface1\"\r\n            Interface \"ns2iface1\"\r\n                type: internal\r\n        Port \"ns1iface2\"\r\n            tag: 33\r\n            Interface \"ns1iface2\"\r\n                type: internal\r\n        Port \"ns2iface3\"\r\n            trunks: [44, 55, 66]\r\n            Interface \"ns2iface3\"\r\n                type: internal\r\n        Port \"testsw1\"\r\n            Interface \"testsw1\"\r\n                type: internal\r\n        Port \"ns2iface2\"\r\n            tag: 33\r\n            Interface \"ns2iface2\"\r\n                type: internal\r\n        Port \"ns1iface1\"\r\n            Interface \"ns1iface1\"\r\n                type: internal<\/pre>\n<p>The new created ports have one line stating with trunks:. This means: The port is dot1q tagged and allows the transport of the tags 44, 55 and 66. All other tags are dropped.<\/p>\n<p>Check if the 3 vlans are working on the new link. Configure also Vlan 77 on the link and verify, that the traffic is dropped at the openvswitch &#8211; reason: Vlan 77 ist not allowed on the port.\u00a0\u00a0Configure an IP address on the native interface nsxiface3 and verify, that this traffic is also dropped.<\/p>\n<p>We have now the following setup:<\/p>\n<div id=\"attachment_155\" style=\"width: 635px\" class=\"wp-caption aligncenter\"><a href=\"http:\/\/www.opencloudblog.com\/wp-content\/uploads\/2013\/09\/ovs-iface3.png\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-155\" class=\"size-full wp-image-155 \" alt=\"Add a port with a restricted vlan list\" src=\"http:\/\/www.opencloudblog.com\/wp-content\/uploads\/2013\/09\/ovs-iface3.png\" width=\"625\" height=\"227\" srcset=\"https:\/\/www.opencloudblog.com\/wp-content\/uploads\/2013\/09\/ovs-iface3.png 625w, https:\/\/www.opencloudblog.com\/wp-content\/uploads\/2013\/09\/ovs-iface3-150x54.png 150w\" sizes=\"(max-width: 625px) 100vw, 625px\" \/><\/a><p id=\"caption-attachment-155\" class=\"wp-caption-text\">After adding a &#8222;trunking&#8220; port with a restricted vlan list<\/p><\/div>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this post I show the basic commands for the openvswitch.\u00a0You should have at least openvswitch 1.9 installed. This is the case, if you are running Ubuntu 13.04. Preparation In this blog, I show more than the basic commands. We use Linux network namespaces to show also the usage on the ip layer. The first [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[],"_links":{"self":[{"href":"https:\/\/www.opencloudblog.com\/index.php?rest_route=\/wp\/v2\/posts\/130"}],"collection":[{"href":"https:\/\/www.opencloudblog.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.opencloudblog.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.opencloudblog.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.opencloudblog.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=130"}],"version-history":[{"count":31,"href":"https:\/\/www.opencloudblog.com\/index.php?rest_route=\/wp\/v2\/posts\/130\/revisions"}],"predecessor-version":[{"id":167,"href":"https:\/\/www.opencloudblog.com\/index.php?rest_route=\/wp\/v2\/posts\/130\/revisions\/167"}],"wp:attachment":[{"href":"https:\/\/www.opencloudblog.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=130"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.opencloudblog.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=130"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.opencloudblog.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=130"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}