VRF & Linux Network Name Space

Introduction

As we know, VRF (Virtual Routing and Forwarding on Switch) and Linux Network Name Space (on Linux hosts) can be used to achieve Network Isolation. Lets see how we can use them together.

Setup 

On the Linux Host there are two namespaces, namely red and blue.
Each of the namespace is connected to its own Linux Bridge (i.e Red Namespace is connected to Bridge_red  and Blue Namespace is connected to Bridge_Blue ). Virtual Interface (veth0) connected to each bridge is assigned the same ip address (10.70.70.12/24). This L3 Network isolation achieved on the same host by using Network Namespaces.


Bridge_red is connected to external interface eth5 via eth5.80(eth5.80 sends tagged packet with vlan value as 80).
Bridge_blue is connected to external interface eth5 via eth5.90(eth5.80 sends tagged packet with vlan value as 90).

Linux host is connected to external device (that has VRF capability). 10.70.70.14/24 is the ip address configured on both the blue and red vrf's.

Commands to Create Red Name Space

//Create a red namespace ip netns add red //Create eth5.80 (80 vlan tagged interface on eth5) ip link add link eth5 eth5.80 type vlan id 80 //Create a veth pair ip link add veth0 type veth peer name veth_red //Set on end of the veth pair to red namespace ip link set veth0 netns red //Bring up the veth pairs ip netns exec red ip link set dev veth0 up ip link set dev veth_red up //create a red bridge and add interfaces brctl addbr bridge_red brctl addif bridge_red eth5.80 brctl addif bridge_red veth_red //bring up bridge interfaces ip link set dev bridge_red up ip link set dev eth5.80 up //Configuer ip address within the namespace ip netns exec red ifconfig veth0 10.70.70.12 netmask 255.255.255.0 up ip netns exec red ip route add default via 10.70.70.12

Commands to Create Blue Name Space ip link add link eth5 eth5.90 type vlan id 90 //Create a blue Namespace. ip netns add blue //Create veth pair ip link add veth0 type veth peer name veth_blue //assign one end of the veth pair to blue namespace ip link set veth0 netns blue //Bring veth pairs ip netns exec blue ip link set dev veth0 up ip link set dev veth_blue up //Create Linux bridge and add interfaces to it. brctl addbr bridge_blue brctl addif bridge_blue eth5.90 brctl addif bridge_blue veth_blue //Bring up the bridge interfaces ip link set dev bridge_blue up ip link set dev eth5.90 up //Assign ip address to veth pair in the blue Namespace ip netns exec blue ifconfig veth0 10.70.70.12 netmask 255.255.255.0 up ip netns exec blue ip route add default via 10.70.70.12

Commands to Create VRF on external device.

//interface connected to eth5 interface GigabitEthernet 3/0/41 switchport switchport mode trunk switchport trunk allowed vlan add 80,90 switchport trunk tag native-vlan spanning-tree shutdown no shutdown ! //Configuration for VRF red rbridge-id 153 vrf red rd 1:1 address-family ipv4 max-route 3600 ! ! ! interface Ve 80 vrf forwarding red ip proxy-arp ip address 10.70.70.14/24 no shutdown ! ! rbridge-id 153 vrf blue rd 1:2 address-family ipv4 max-route 3600 ! ! ! interface Ve 90 vrf forwarding blue ip proxy-arp ip address 10.70.70.14/24 no shutdown ! !

Communication from Linux host to external device (ie Network Namespace to VRF)

Here ping is initiated from the veth0(10.70.70.12) in the blue namespace to 10.70.70.14 (in the blue VRF)

Here ping is initiated from the veth0(10.70.70.12) in the red namespace to 10.70.70.14 (in the redVRF)

Tcpdump can be used on LinuxBridge interfaces to verify that traffic is indeed flowing to the correct VRF on the external device.

Network Isolation (L3,ip address isolation)  can be achieved by using Linux Network Namespaces and VRFs.Also, we have seen here how they can be interconnected.

Network Isolation is an important concept in multi-tenant cloud environment.

Security Groups

Introduction

Security Groups are like ip filter rules that are applied to an instances networking.When an instance is launched one more Security Groups are applied on it. The rules within the security groups determine what traffic is allowed for that instance. Rules from all Security Groups associated to the instance are evaluated to determine if traffic can be allowed to or from that instance.

Setup 

A network by the name "test_net" having two VMS attached to it namely "test_1" and "test_2"

We are using LinuxBridge with VLAN Networking enabled. 

The output from "brctl-show" is as follows

brctl show bridge name bridge id STP enabled interfaces brq51afd1d5-f4 8000.4ecb944b6f31 no eth1.82 tap320b515f-92 tap3ab26719-5b tap4b4a6d8d-3a tapc4882759-84

Creation of Security Group

Create a security-group having rules to allow all kind of traffic in both directions (ingress,egress).

neutron security-group-create TEST_SEC_GROUP_ALLOW_ALL neutron security-group-rule-create --direction ingress TEST_SEC_GROUP_ALLOW_ALL neutron security-group-rule-create --direction egress TEST_SEC_GROUP_ALLOW_ALL

Associate security group with VM instances

Now associate "TEST_SEC_GROUP_ALLOW_ALL" to both the VM instances test_1 and test_2
root@os-157:~# nova add-secgroup test_1 TEST_SEC_GROUP_ALLOW_ALL root@os-157:~# nova show test_1 +--------------------------------------+----------------------------------------------------------------+ | Property | Value | +--------------------------------------+----------------------------------------------------------------+ | status | ACTIVE | | updated | 2014-02-12T06:58:02Z | | OS-EXT-STS:task_state | None | | key_name | None | | image | cirros-0.3.1-x86_64-uec (ad14252a-0a2f-4c1b-ac20-bfa6fa00f1c7) | | hostId | fbc3d9b727547d7a7363f47a40a082c8b0ddcb981ab7bfe286b20ee4 | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2014-02-12T06:33:35.000000 | | flavor | m1.nano (42) | | id | 007245b1-5a9c-433c-a05c-5634a10657b0 | | security_groups | [{u'name': u'TEST_SEC_GROUP_ALLOW_ALL'}] | | OS-SRV-USG:terminated_at | None | | user_id | 600f7ff30a7441b58d3a09bb5e62f736 | | name | test_1 | | created | 2014-02-12T06:33:30Z | | tenant_id | 693c35b741514e4780998bca22485a02 | | test network | 9.9.9.2 | | OS-DCF:diskConfig | MANUAL | | metadata | {} | | os-extended-volumes:volumes_attached | [] | | accessIPv4 | | | accessIPv6 | | | progress | 0 | | OS-EXT-STS:power_state | 1 | | OS-EXT-AZ:availability_zone | nova | | config_drive | | +--------------------------------------+----------------------------------------------------------------+ root@os-157:~# nova add-secgroup test_2 TEST_SEC_GROUP_ALLOW_ALL root@os-157:~# nova show test_2 +--------------------------------------+----------------------------------------------------------------+ | Property | Value | +--------------------------------------+----------------------------------------------------------------+ | status | ACTIVE | | updated | 2014-02-12T06:34:35Z | | OS-EXT-STS:task_state | None | | key_name | None | | image | cirros-0.3.1-x86_64-uec (ad14252a-0a2f-4c1b-ac20-bfa6fa00f1c7) | | hostId | fbc3d9b727547d7a7363f47a40a082c8b0ddcb981ab7bfe286b20ee4 | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2014-02-12T06:34:35.000000 | | flavor | m1.nano (42) | | id | 983ada86-4739-43f3-8cdb-8815dce3230f | | security_groups | [{u'name': u'TEST_SEC_GROUP_ALLOW_ALL'}] | | OS-SRV-USG:terminated_at | None | | user_id | 600f7ff30a7441b58d3a09bb5e62f736 | | name | test_2 | | created | 2014-02-12T06:34:31Z | | tenant_id | 693c35b741514e4780998bca22485a02 | | test network | 9.9.9.4 | | OS-DCF:diskConfig | MANUAL | | metadata | {} | | os-extended-volumes:volumes_attached | [] | | accessIPv4 | | | accessIPv6 | | | progress | 0 | | OS-EXT-STS:power_state | 1 | | OS-EXT-AZ:availability_zone | nova | | config_drive | | +--------------------------------------+----------------------------------------------------------------+

Communication between the two VM instances 

Ping as well as ssh is possible between the two VM instances as the applied security group is the least restrictive.

Security group that allows only SSH

Create security group that allows only incoming SSH traffic, associate it to both VM instances.

Note: Remove the earlier security group from the VM instances. SSH means TCP on port 22.

neutron security-group-create TEST_SEC_GROUP_ALLOW_ONLY_SSH neutron security-group-rule-create --direction egress TEST_SEC_GROUP_ALLOW_ONLY_SSH neutron security-group-rule-create --direction ingress --protocol tcp --port-range-min 22 --port-range-max 22 TEST_SEC_GROUP_ALLOW_ONLY_SSH root@os-157:~# nova remove-secgroup test_2 TEST_SEC_GROUP_ALLOW_ALL root@os-157:~# nova remove-secgroup test_1 TEST_SEC_GROUP_ALLOW_ALL root@os-157:~# root@os-157:~# root@os-157:~# nova add-secgroup test_1 TEST_SEC_GROUP_ALLOW_ONLY_SSH root@os-157:~# nova add-secgroup test_2 TEST_SEC_GROUP_ALLOW_ONLY_SSH

Communication between VM's

We observe that VM's can communicate only using SSH, ping between the VM's fails.

Under the hood

"IP tables" are utilized to enforce Security Groups on the tap interfaces connected to the VM instances.
Here the rule "-A neutron-linuxbri-i3ab26719-5 -p tcp -m tcp --dport 22 -j RETURN"  accepts TCP traffic on destination port 22(SSH)

root@os-157:~# iptables -S | grep tap3ab26719-5b -A neutron-linuxbri-FORWARD -m physdev --physdev-out tap3ab26719-5b --physdev-is-bridged -j neutron-linuxbri-sg-chain -A neutron-linuxbri-FORWARD -m physdev --physdev-in tap3ab26719-5b --physdev-is-bridged -j neutron-linuxbri-sg-chain -A neutron-linuxbri-INPUT -m physdev --physdev-in tap3ab26719-5b --physdev-is-bridged -j neutron-linuxbri-o3ab26719-5 -A neutron-linuxbri-sg-chain -m physdev --physdev-out tap3ab26719-5b --physdev-is-bridged -j neutron-linuxbri-i3ab26719-5 -A neutron-linuxbri-sg-chain -m physdev --physdev-in tap3ab26719-5b --physdev-is-bridged -j neutron-linuxbri-o3ab26719-5 root@os-157:~# iptables -S | grep neutron-linuxbri-i3ab26719-5 -N neutron-linuxbri-i3ab26719-5 -A neutron-linuxbri-i3ab26719-5 -m state --state INVALID -j DROP -A neutron-linuxbri-i3ab26719-5 -m state --state RELATED,ESTABLISHED -j RETURN -A neutron-linuxbri-i3ab26719-5 -p tcp -m tcp --dport 22 -j RETURN -A neutron-linuxbri-i3ab26719-5 -s 9.9.9.3/32 -p udp -m udp --sport 67 --dport 68 -j RETURN -A neutron-linuxbri-i3ab26719-5 -j neutron-linuxbri-sg-fallback -A neutron-linuxbri-sg-chain -m physdev --physdev-out tap3ab26719-5b --physdev-is-bridged -j neutron-linuxbri-i3ab26719-5

VLAN Networking OpenStack

Introduction

Here, We will look at how to setup a network (based on VLAN). Connect VM instances to these networks. Make necessary configurations so that VM's can communicate with each other across different networks. We also try to figure out how the Linux Bridge is setup as we configure the Network and the Virtual Machines.

This would be extended later using an external switch/router in the subsequent blog.

Read this blog to understand about Linux Network Namespaces

Create Network.

Created a new network named "application_net" having subnet "103.103.103.0/24" and gateway ip address as "103.103.103.1". VM's created on this network would have ip address allocated from this subnet.

user@os-157:~$ neutron net-create application_net Created a new network: +----------------+--------------------------------------+ | Field | Value | +----------------+--------------------------------------+ | admin_state_up | True | | id | 5cc1629c-7d97-4323-a92c-42ccd6d0dfe5 | | name | application_net | | shared | False | | status | ACTIVE | | subnets | | | tenant_id | d8fde3204bbd43258cb0e1bb5ee2502e | +----------------+--------------------------------------+ user@os-157:~$ neutron subnet-create application_net 103.103.103.0/24 --name application_sub --gateway 103.103.103.1 Created a new subnet: +------------------+------------------------------------------------------+ | Field | Value | +------------------+------------------------------------------------------+ | allocation_pools | {"start": "103.103.103.2", "end": "103.103.103.254"} | | cidr | 103.103.103.0/24 | | dns_nameservers | | | enable_dhcp | True | | gateway_ip | 103.103.103.1 | | host_routes | | | id | 99e0f713-0024-45d8-8f50-c4d74379faf8 | | ip_version | 4 | | name | application_sub | | network_id | 5cc1629c-7d97-4323-a92c-42ccd6d0dfe5 | | tenant_id | d8fde3204bbd43258cb0e1bb5ee2502e | +------------------+------------------------------------------------------+

List the Network and Subnet details.

Use neutron commands to see the created and network and subnet-details.

user@os-157:~$ neutron net-list +--------------------------------------+-----------------+-------------------------------------------------------+ | id | name | subnets | +--------------------------------------+-----------------+-------------------------------------------------------+ | 5cc1629c-7d97-4323-a92c-42ccd6d0dfe5 | application_net | 99e0f713-0024-45d8-8f50-c4d74379faf8 103.103.103.0/24 | +--------------------------------------+-----------------+-------------------------------------------------------+ user@os-157:~$ neutron subnet-list +--------------------------------------+-----------------+------------------+------------------------------------------------------+ | id | name | cidr | allocation_pools | +--------------------------------------+-----------------+------------------+------------------------------------------------------+ | 99e0f713-0024-45d8-8f50-c4d74379faf8 | application_sub | 103.103.103.0/24 | {"start": "103.103.103.2", "end": "103.103.103.254"} | +--------------------------------------+-----------------+------------------+------------------------------------------------------+

Create two Virtual Machines  connected to the Network.

Create two VM's "app_1" and "app_2" connected to this newly created Network "application_net"

nova boot --flavor 1 --image b072e4b1-ea4e-4e3f-8741-6bce8f34eb40 --security_group default app_1 nova boot --flavor 1 --image b072e4b1-ea4e-4e3f-8741-6bce8f34eb40 --security_group default app_2

Bridge details and Network Topology.

Let's see what all has happened under the hood.

A new bridge(Linux Bridge) is created by the name "brq5cc1629c-7d" and there are tap/veth interfaces that gets attached to the bridge.There is tap interface for that gets attached to the bridge for every VM in the network.(application_net)

user@os-157:~$ brctl show bridge name bridge id STP enabled interfaces brq5cc1629c-7d 8000.aa1c8f3ee486 no eth1.85 tap0f9f4744-3f tap7ed0efa1-d1 tapdfcfc52e-66

Communication between two VM's

The two VM's app_1 and app_2 will be able to ping each other, as they are on the same network (application_network)

Ping between VM's in application_net

Create another network and VMs

Lets create another network "db_net" with  VMS "db_1" and "db_2"  in that network.

These two VM's would be able to talk to each other. (as they are on the same Linux bridge).

Communication between VM's between the two networks

VM (app_1) in application_network will not be able to ping VM(db_1) in db_network.
In order to achieve such a communication, the two bridges should be connected to a router(Physical/Virtual)

user@os-157:~$ neutron router-interface-add router1 56d0c029-c9db-457f-968f-de9148a7756e Added interface c86010bd-e4b8-4686-b031-100321489d32 to router router1. neutron router-interface-add router1 99e0f713-0024-45d8-8f50-c4d74379faf8 Added interface c6283960-5d3f-46a7-b132-f836952a0d9c to router router1. user@os-157:~$ sudo ip netns exec qrouter-7ce1bc78-ae3f-488f-8dc8-f1869bd905d4 ifconfig lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) qg-82c8613e-77 Link encap:Ethernet HWaddr fa:16:3e:cf:ca:3b inet addr:172.24.4.226 Bcast:172.24.4.239 Mask:255.255.255.240 inet6 addr: fe80::f816:3eff:fecf:ca3b/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:2534 errors:0 dropped:2405 overruns:0 frame:0 TX packets:15 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:595684 (595.6 KB) TX bytes:846 (846.0 B) qr-c6283960-5d Link encap:Ethernet HWaddr fa:16:3e:a7:e4:12 inet addr:103.103.103.1 Bcast:103.103.103.255 Mask:255.255.255.0 inet6 addr: fe80::f816:3eff:fea7:e412/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:22 errors:0 dropped:1 overruns:0 frame:0 TX packets:9 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:5052 (5.0 KB) TX bytes:594 (594.0 B) qr-c86010bd-e4 Link encap:Ethernet HWaddr fa:16:3e:27:58:6b inet addr:15.15.15.1 Bcast:15.15.15.255 Mask:255.255.255.0 inet6 addr: fe80::f816:3eff:fe27:586b/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:25 errors:0 dropped:2 overruns:0 frame:0 TX packets:9 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:5391 (5.3 KB) TX bytes:594 (594.0 B)

Once the two bridges are connected to the router, the two VMS (from different networks) would be able to ping each other.

Linux Network NameSpaces

Introduction

Namespace : A class of elements (e.g. addresses, file locations, etc.) in which each element has a name unique to that class, although it may be shared with elements in other classes. (Definition)

Generally, Linux shares a single set of Network interfaces and routing tables across the entire Operating System. With Networking Namespace, its now possible for each Namespace to have its own routing table and other networking constructs.

Creating a Network Namespace

Let's create two Namespaces "red" and "blue". root@controller:~#ip netns add red root@controller:~#ip netns add blue

Listing the Namespace

root@controller:~# ip netns list blue red

Executing a command in a Namespace

ip netns exec <namespace> <command> root@controller:~# ip netns exec red ip link list 1: lo: <loopback> mtu 65536 qdisc noop state DOWN mode DEFAULT link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

Adding interfaces to a Namespace

Its not possible to add a physical interface(like eth0) to a Namespace. One needs to create a virtual interface and add it to the Namespace. Virtual interfaces are created in pairs. One of them can be added to the default Namespace and other to the newly created Namespace.

Creating pair of Virtual Interface

Lets create two virtual interfaces "veth0" and "veth_blue". virtual interfaces are created here in the default namespace.

root@controller:~# ip link add veth0 type veth peer name veth_blue root@controller:~# ip link list 10: veth_blue: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000 link/ether 02:9e:3d:c4:5d:05 brd ff:ff:ff:ff:ff:ff 11: veth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000 link/ether d2:e3:09:bf:41:10 brd ff:ff:ff:ff:ff:ff

Adding Virtual Interface to a Namespace

root@controller:~# ip link set veth0 netns blue

This command adds veth0 to Namespace blue, the other pair (veth_blue) is still present in the default namespace

Listing of default Namespace

This commands show the listing of default namespace, veth_blue is still present in the default namespace.

root@controller:~# ip link list 10: veth_blue: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000 link/ether 02:9e:3d:c4:5d:05 brd ff:ff:ff:ff:ff:ff

Listing of blue Namespace

veth0 has now moved to the blue Namespace.

root@controller:~# ip netns exec blue ip link list 11: veth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000 link/ether d2:e3:09:bf:41:10 brd ff:ff:ff:ff:ff:ff

Similar setup for Red Namespace

Adding veth0 to red Namespace and its peer (veth_red) remains in the default namespace.

ip link add veth0 type veth peer name veth_red ip link set veth0 netns red