{"id":614,"date":"2016-01-09T22:42:18","date_gmt":"2016-01-09T21:42:18","guid":{"rendered":"http:\/\/www.opencloudblog.com\/?p=614"},"modified":"2021-01-17T14:48:11","modified_gmt":"2021-01-17T13:48:11","slug":"openstack-juno-neutron-deployment-part-2-network-infrastructure","status":"publish","type":"post","link":"https:\/\/www.opencloudblog.com\/?p=614","title":{"rendered":"OpenStack Liberty Neutron Deployment (Part 2 Network Infrastructure)"},"content":{"rendered":"<h1>The physical network<\/h1>\n<p>A cloud needs an &#8222;physical&#8220; network using real &#8222;switching routers&#8220; to transport data between the physical nodes.<\/p>\n<p>The physical network has the following layout:<\/p>\n<div id=\"attachment_589\" style=\"width: 961px\" class=\"wp-caption aligncenter\"><a href=\"http:\/\/www.opencloudblog.com\/wp-content\/uploads\/2015\/01\/cloudphysnet.png\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-589\" class=\"size-full wp-image-589\" src=\"http:\/\/www.opencloudblog.com\/wp-content\/uploads\/2015\/01\/cloudphysnet.png\" alt=\"Physical set up\" width=\"951\" height=\"419\" srcset=\"https:\/\/www.opencloudblog.com\/wp-content\/uploads\/2015\/01\/cloudphysnet.png 951w, https:\/\/www.opencloudblog.com\/wp-content\/uploads\/2015\/01\/cloudphysnet-150x66.png 150w\" sizes=\"(max-width: 951px) 100vw, 951px\" \/><\/a><p id=\"caption-attachment-589\" class=\"wp-caption-text\">Physical set up<\/p><\/div>\n<p>Each floating pool is implemented using one vlan on the physical network infrastructure. Each flat network requires also one vlan on the physical network. VXLAN requires also one vlan on the physical network.<\/p>\n<p>VXLAN adds additional headers (IP and UDP) to packets. You must increase the mtu of the transport VXLAN vlan to a higher value, let&#8217;s take 1600. Do not follow the &#8222;recommendations&#8220; to reduce the MTU of VMs using the DHCP server. Such a &#8222;recommendation&#8220; only leads to big problems in a production environment &#8212; <strong>do not use a reduced MTU<\/strong>!<\/p>\n<p>The configuration of the physical switching router could be (using a pseudo config language):<\/p>\n<pre class=\"lang:sh decode:true\" title=\"Switching Router pseudo config\">vlan 100 name Floating-Pool-1\nvlan 101 name Floating-Pool-2\nvlan 200 name Flat-Net-1\nvlan 201 name Flat-Net-2\n# set the mtu to 1600 for vlan 4000\nvlan 4000 name vxlan mtu 1600\n# do not use vlan 1 for untagged packets\nvlan 4090 name native vlan\n#\n############\n#\ninterface vlan 100\n  description Floating-Network-1\n  ip address 198.18.0.1\/20\ninterface vlan 101\n  description Floating-Network-2\n  ip address 198.18.16.1\/20\ninterface vlan 200\n  description Flat-Network-1\n  ip address 198.19.1.1\/24\ninterface vlan 201\n  description Flat-Network-2\n  ip address 198.19.2.1\/24\n# a L3 interface for the vxlan vlan may be added\n#\n#############\n#\n# the ports to the nodes (network and compute)\n# They use the same config !\n# just one link to each node - multiple links using LACP may also be used\ninterface port1\n  description to-network-node\n  mode trunk\n  trunk native vlan 4090\n  trunk vlan 100,101,200,201,4000,4090\n  mtu 1600\ninterface port2\n  description to-compute-node\n  mode trunk\n  trunk native vlan 4090\n  trunk vlan 100,101,200,201,4000,4090\n  mtu 1600<\/pre>\n<h1>The network set up of the nodes<\/h1>\n<p>The basic network set up of compute and network nodes is exactly the same. On the nodes, a basic network configuration of the Operating system must be provided by the user. This configuration connects the transport network for VMs on the nodes to the physical network. The network setup of a node is shown in the drawing below:<\/p>\n<div id=\"attachment_603\" style=\"width: 769px\" class=\"wp-caption aligncenter\"><a href=\"http:\/\/www.opencloudblog.com\/wp-content\/uploads\/2015\/01\/cloudphysnode1.png\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-603\" class=\"size-full wp-image-603\" src=\"http:\/\/www.opencloudblog.com\/wp-content\/uploads\/2015\/01\/cloudphysnode1.png\" alt=\"Openvswitch Bridges on the Nodes\" width=\"759\" height=\"661\" srcset=\"https:\/\/www.opencloudblog.com\/wp-content\/uploads\/2015\/01\/cloudphysnode1.png 759w, https:\/\/www.opencloudblog.com\/wp-content\/uploads\/2015\/01\/cloudphysnode1-150x131.png 150w, https:\/\/www.opencloudblog.com\/wp-content\/uploads\/2015\/01\/cloudphysnode1-689x600.png 689w\" sizes=\"(max-width: 759px) 100vw, 759px\" \/><\/a><p id=\"caption-attachment-603\" class=\"wp-caption-text\">Openvswitch Bridges on the Nodes<\/p><\/div>\n<p>The cool part of the network setup is, that only ONE physical interface (or a LACP bundle) is used to transport VM traffic. This interface is connected to br-uplink. If you read the Openstack manuals and believe in everything which is written there, you would end up using FOUR physical interfaces:<\/p>\n<ul>\n<li>one physical interface for VXLAN transport. This is not true &#8211; VXLAN transport requires an IP address in the default network namespace of the operating system. We choose an Openvswitch internal port on the Openvswitch bridge br-uplink and attach an IP address to it. The default MTU must be set to a higher value. We use 1600.<\/li>\n<li>one link to the physical network via br-vlan (or br-eth1 in the Openstack documentation) using a physical interface for the Openstack ML2 &#8222;vlan&#8220; type driver. This is not true &#8211; we take an Openvswitch patch port as the uplink. This port connects br-vlan to be use by the Openstack ML2 mechanism driver to br-uplink.<\/li>\n<li>two interfaces to map the two floating pools to two L3 agents, each using an Openvswitch bridge, which is connected to the physical network using a physical ethernet interface. This is not true &#8211; L3 agents are clever enough to avoid the usage of any additional bridges. And one L3 agent may manage multiple floating pools (or external networks if you are talking &#8222;Openstack&#8220;).<\/li>\n<\/ul>\n<p>A part of the network configuration must be provided by the admin of the nodes during system startup. This configuration must be placed in the network configuration of the Operating system. The configuration depends on the Linux distribution.<\/p>\n<p>The shell code for the network setup is (the management network is omitted):<\/p>\n<pre class=\"lang:sh decode:true\" title=\"Shell code to install the base networking for Openstack Nodes\">#\n# the bridge, which connects the nodes to the transport network\novs-vsctl add-br br-uplink\n# the bridge used by Openstack Neutron to connect vlans and flatdhcp networks\novs-vsctl add-br br-vlan\n# the integration bridge used by Openstack\novs-vsctl add-br br-int\n#\n# add the uplink (with dot1q tags 101,102,...)\n# we assume, that eth1 is the uplink interface\nip link set dev eth1 up\n# set the mtu of the physical uplink to the switch\nip link set dev eth1 mtu 1600\n#\n# disable gro and lro !! on the uplink\nethtool -K eth1 gro off\nethtool -K eth1 lro off\n#\n# enable for intel NICs udp port hashing to distribute traffic to different queues\nethtool -N eth1 rx-flow-hash udp4 sdfn\n#\novs-vsctl add-port br-uplink eth1 -- set port eth1 vlan_mode=trunk trunk=100,101,200,201,4000\n#\n# patch ports between br-uplink and br-vlan\novs-vsctl add-port br-vlan patch-to-uplink -- set Interface patch-to-uplink type=patch options:peer=patch-to-vlan\novs-vsctl add-port br-uplink patch-to-vlan -- set Interface patch-to-vlan type=patch options:peer=patch-to-uplink\n#\n# !! on br-uplink the allowed vlan tags on the patch port from br-vlan must be filtered using Openflow rules\n# !! if this is not done, there is a risk that vlans from the infrastructure may get mixed with local vlans\n# !! of br-int, if the neutron Openvswitch agent fails to set up the vlan mapping on br-vlan or br-int\n# TBD\n###\n# create the Linux IP interface required for VXLAN transport\n# this interface is attached to vlan 4000 of br-uplink\n# XXX = Last octet of the VXLAN interface ip address of the node\novs-vsctl add-port br-uplink l3vxlan tag=4000 -- set Interface l3vxlan type=internal\nip addr add 10.255.255.XXX\/24 dev l3vxlan\nip link set dev l3vxlan up\n# set the mtu of the logical vxlan interface\nip link set dev l3vxlan mtu 1600\n<\/pre>\n<p><strong>Do not enable ip_forwarding<\/strong> on any of the nodes. This is not necessary, even if the Openstack documentation declares this as a prerequisite.<\/p>\n<p>Neutron in Openstack Liberty does not require to use br-ex to get a functional L3 agent providing router services. The default setup in the Openstack documentation is using br-ex, but there are other ways to implement networking, e.g. the one shown in this article series.<\/p>\n<p><a title=\"Part 3\" href=\"http:\/\/www.opencloudblog.com\/?p=630\">Continue reading (part 3)<\/a> the Openstack Neutron setup<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The physical network A cloud needs an &#8222;physical&#8220; network using real &#8222;switching routers&#8220; to transport data between the physical nodes. The physical network has the following layout: Each floating pool is implemented using one vlan on the physical network infrastructure. Each flat network requires also one vlan on the physical network. VXLAN requires also one [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[14],"tags":[],"_links":{"self":[{"href":"https:\/\/www.opencloudblog.com\/index.php?rest_route=\/wp\/v2\/posts\/614"}],"collection":[{"href":"https:\/\/www.opencloudblog.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.opencloudblog.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.opencloudblog.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.opencloudblog.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=614"}],"version-history":[{"count":22,"href":"https:\/\/www.opencloudblog.com\/index.php?rest_route=\/wp\/v2\/posts\/614\/revisions"}],"predecessor-version":[{"id":825,"href":"https:\/\/www.opencloudblog.com\/index.php?rest_route=\/wp\/v2\/posts\/614\/revisions\/825"}],"wp:attachment":[{"href":"https:\/\/www.opencloudblog.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=614"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.opencloudblog.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=614"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.opencloudblog.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=614"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}