Openstack devstack setup with VXLAN as overlay

Good News is that OVS now supports VXLAN tunnel protocol. This makes opensource Openflow Controllers equipped with power to get ready enter into new markets where Overlay networks are preferred. Here is the release notes for Openvswitch release 1.10.

I wanted to build a Openstack setup with VXLAN and try it out myself. But there was no proper documentation put together. Hope this post helps folks who want to build VXLAN overlay network based Openstack setup.

Setup:

I have two Ubuntu Servers and a Windows desktop. I used Ubuntu Servers as compute nodes , twos VMs on virtual box in Windows. One VM as a Controller and second VM as a Network node.

Since VXLAN support in OVS is in master branch, I chose to use devstack to setup Openstack.

For simplicity I will not mention second Compute Node in the config section as the settings are same as other. And each system has only one NIC.

Controller: 192.168.1.121
Compute Node: 192.168.1.112
Network Node: 192.168.1.123

localrc for Controller:

#SCHEDULER=nova.scheduler.simple.SimpleScheduler
SCHEDULER=nova.scheduler.filter_scheduler.FilterScheduler
LOGFILE=/opt/stack/data/stack.log
SCREEN_LOGDIR=/opt/stack/data/log
RECLONE=yes
#disable_service n-net, n-cpu
#enable_service q-svc, q-agt, q-l3, q-meta, q-dhcp, neutron
ENABLED_SERVICES=g-api,g-reg,key,n-api,n-crt,n-obj,n-cond,cinder,c-sch,c-api,c-vol,n-sch,n-novnc,n-xvnc,n-cauth,horizon,rabbit,mysql,neutron,q-svc
Q_SRV_EXTRA_OPTS=(tenant_network_type=vxlan)
Q_AGENT_EXTRA_AGENT_OPTS=(tunnel_types=vxlan vxlan_udp_port=8472)
ENABLE_TENANT_TUNNELS=True

localrc for compute

ENABLED_SERVICES=n-cpu,rabbit,neutron,q-agt
LOGFILE=/opt/stack/data/stack.log
SCREEN_LOGDIR=/opt/stack/data/log
RECLONE=yes
# Openstack services running on controller node
SERVICE_HOST=192.168.1.121 # replace this with the IP address of the controller node
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
Q_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
Q_AGENT_EXTRA_AGENT_OPTS=(tunnel_types=vxlan vxlan_udp_port=8472)
Q_SRV_EXTRA_OPTS=(tenant_network_type=vxlan)
ENABLE_TENANT_TUNNELS=True 

localrc for network node

SERVICE_HOST=192.168.1.121
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
Q_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
#SCHEDULER=nova.scheduler.simple.SimpleScheduler
SCHEDULER=nova.scheduler.filter_scheduler.FilterScheduler
LOGFILE=/opt/stack/data/stack.log
SCREEN_LOGDIR=/opt/stack/data/log
RECLONE=yes
ENABLED_SERVICES=q-agt,q-l3,q-dhcp,q-meta,rabbit
Q_SRV_EXTRA_OPTS=(tenant_network_type=vxlan)
Q_AGENT_EXTRA_AGENT_OPTS=(tunnel_types=vxlan vxlan_udp_port=8472)
ENABLE_TENANT_TUNNELS=True

Other Changes

As expected, this does not work straight forward. Openstack requires us to install OVS manually as it is not Officially released by Ubuntu.

Download OVS 1.10 from here on compute nodes and Network node. 

Here are the installation instructions

./configure --prefix=/usr --localstatedir=/var  --with-linux=/lib/modules/`uname -r`/build
make
make install
sudo rmmod openvswitch
sudo  insmod datapath/linux/openvswitch.ko
sudo mkdir -p /usr/etc/openvswitch
sudo pkill ovsdb-tool
sudo pkill ovsdb-server
sudo pkill ovs-vswitchd
sudo rm -rf /usr/etc/openvswitch/conf.db
sudo  ovsdb-tool create /usr/etc/openvswitch/conf.db vswitchd/vswitch.ovsschema
sudo ovsdb-server --remote=punix:/var/run/openvswitch/db.sock \
                     --remote=db:Open_vSwitch,manager_options \
                     --private-key=db:SSL,private_key \
                     --certificate=db:SSL,certificate \
                     --bootstrap-ca-cert=db:SSL,ca_cert \
                     --pidfile --detach

sudo ovs-vsctl --no-wait init
sudo  ovs-vswitchd --pidfile --detach
sudo ovs-vsctl add-br br-int

One more issues I faced is the VNC console. Devstack multi node scripts seems to have problem. They dont generate proper config for Compute Nodes. The listen address is set to 127.0.0.1 and few variables not set. I changed the nova.conf and restarted nova-compute.

novnc_enabled=True
novncproxy_base_url=http://192.168.1.121:6080/vnc_auto.html
xvpvncproxy_base_url=http://192.168.1.121:6081/console
novncproxy_port=6080
vncserver_proxyclient_address=192.168.1.112
vncserver_listen=0.0.0.0

Hope, you find this post useful. Let me know in comments if you need more information.

Setting up Dev environment for Nodejs coffeescript development

This article describes my experiences in selecting the development environment for Nodejs and coffee scripts.

First Step: Choosing  OS

Most of the JS developers use windows. Since I am from networking background, I preferred Linux and that too Ubuntu system. Download and installation of Ubuntu is simple and fast.

Second step: Choosing an Editor

I do not have Java Background. In my career so far, I used vim, cscope and ctags. vim by default does not highlight coffee script syntax. When opened an coffeescript file, it showed as if its plain text. Hence I began to search for proper editor to use. I understand there are some very good IDE like eclipse. But I have to jump start and not worry about learning new IDEs.

I found one good vim extension that suits my needs. It is vim-coffee-script. Here is a quick setup guide installing it. If you are used to other editors – Check this link if it needs any extensions and add your experiences in comments of this blog.

Here are the steps to install vim-coffee-script:

1) Download the latest zipball from vim.org or github. The latest version on github is under Download Packages (don’t use the Download buttons.)

2) Extract the archive into ~/.vim/:

unzip -od ~/.vim vim-coffee-script-HASH.zip

You can compile coffee scripts using vim-coffee-scripts using vim. Read more about it here.

Third Step: Installing node and npm

Ubuntu has good package support. To install node, coffee script and npm just run the following.

sudo apt-get install node coffeescript npm
Thats it. Development environment for Node using coffeescripts in Ubuntu is ready to use.