TheRuddyDuck

Configure Hortonworks HDFS integration with EMC ViPR

ViPR HDFS is a POSIX-like Hadoop compatible file system (HCFS) that enables you to run Hadoop 2.0 applications on top of your ViPR storage infrastructure. You can configure your Hadoop distribution to run against the built-in Hadoop file system against ViPR HDFS, or any combination of HDFS, ViPR HDFS, or other HCFSs available in your environment. When you run Hadoop applications on top of ViPR, the ViPR HDFS software manages both the metadata and the data access so the Hadoop NameNode and Hadoop data nodes do not perform those functions.

The core-site.xml contains configuration information about HDFS. For Hadoop to use ViPR for HDFS requires configuration of the core-site.xml file. The latest release of ViPR integrates data services API with Hadoop, by instantiating a java client jar file on each worker node as a Hadoop Compatible File System (HCFS). The jar file is placed in the hadoop classpath on each node.

Modifying the core-site.xml to add the ViPRFS java classes, defining the ViPRFS URI, allows Hadoop cluster’s to point to the ViPR HDFS implementation. The MapReduce jobs can then read and write data directly on the ViPRFS file system.

By using EMC ViPR for as a HCFS for Cloudera, the local HDFS service, including NameNodes and DataNodes services do not need to be used. Once the custom core-site.xml is configured, the local HDFS service will be disabled.

For the configuration examble delow the fs.defaultFS is viprfs://ObjectBucket.ObjectTenant.HSK

In a  previous  post I showed how to setup object data services within ViPR.  Building on that example we would create a tenant called "ObjectTenant" that has access to the object data store. We would then create a bucket from the ViPR service catalog called "ObjectBucket" that supplies both s3 and HDFS access to the object data store (Example of bucket creation here). Finally "HSK" in the defaultFS name is used per ViPR instance. Meaning that a single core-site.xml could be configured to access mutliple ViPR data services instances.

the Table below shows the configuration that need to be place into the core-site.xml file

Property	Description
<property> <name>fs.defaultFS</name> <value>viprfs://ObjectBucket.ObjectTenant.HSK</value> </property>	File system default name ObjectBucket = The name of the bucket ObjectTenant = The tenant name HSK = FS installation name defined in the next property
<property> <name>fs.vipr.installations</name> <value>HSK</value> </property>	Add this custom ViPR HDFS property as a comma-separated list of names. The names are further defined by the fs.vipr.installation. [installation_name].hosts property to uniquely identify sets of ViPR data nodes. The names are used as a component of the authority section of the ViPR HFDS file system URI.
<property> <name>fs.vipr.installation.HSK.hosts</name> <value>10.10.81.160</value> </property	Add this custom ViPR HDFS property specifying the IP addresses of the ViPR cluster's data nodes or the load balancers for each name listed in the fs.vipr.installations property. Specify the value in the form of a comma-separated list of IP addresses.ViPR data services IP address. Only other value that needs to be changed is “HSK”, which is defined in the previous property and the Data Services IP address. Ip address can also be FQDN.
<property> <name>fs.viprfs.impl</name> <value>com.emc.hadoop.fs.vipr.ViPRFileSystem</value> </property>	Custom ViPR file system property. No changes necessary
<property> <name>fs.AbstractFileSystem.viprfs.impl</name> <value>com.emc.hadoop.fs.vipr.ViPRAbstractFileSystem</value> </property	Custom ViPR file system property. No changes necessary
<property> <name>fs.permissions.umask-mode</name> <value>000</value> </property>	Custom ViPR file system property. No changes necessary
<property> <name>fs.viprfs.auth.anonymous_translation</name> <value>CURRENT_USER</value> </property>	Custom ViPR file system property. No changes necessary

ViPR custom Core-site.xml properties and values

Hadoop distribution	ViPR HDFS JAR
Pivotal	hadoop-2.0.x-alpha-viprfs-1.0.1.jar
Cloudera	hadoop-2.0.x-alpha-viprfs-1.0.1.jar
Hortonworks	hadoop-2.2.viprfs-1.0.1.jar

ViPR Java client version

This section will show you how to use Hortonwoks Ambari Server to push a new core-site.xml file to all the nodes that make up your HDP cluster. For Hortonworks the best way to push this configuration to the nodes is to use the Hortonworks Ambari Server GUI.

On the dashboard screen of the Ambari server click the HDFS service in the left panel

When the service page opens up, click Configs

Scroll down the page until you see the “Custom core-site.xml” tab. Click on it to expand.

Click Add Property

Add a property name and value for each property found in table 4. Don not add fs.defaultFS property here. That will be done in the next step. The screen shot above shows the ViPR custom properties and values for this guide.

Click on the Advance tab above the Custom core-site.xml tab.

g For the fs.defaultFS property enter the value for bucket, tenant, and FS installation name.

Save your changes by clicking Save in the bottom right corner.

Configuration is saved and the new core-site.xml is pushed to all the managed clients.

On the dashboard the HDFS, YARN, and MapReduce2 service all show as being needed to be rebooted. ViPR HDFS software manages both the metadata and the data access so the Hadoop NameNode and Hadoop data nodes do not perform those functions. There is no restart button so stop then start the services.

From the dashboard click on the HDFS service. Click on Stop. Repeat for the YARN and MapReduce services

From the dashboard click on the HDFS service and click Start. Service start will fail, but will push client config to all nodes. Start Mapreduce and YARN service.

From one of the HDP nodes verify the core-site.xml file using the cat and grep command cat /etc/hadoop/conf/core-site.xml |grep HSK

Upload the ViPR java client to the HDPManager. Use SCP to copy the file to all HDP nodes. Place it in the hadoop classpath directory /usr/lib/hadoop-hdfs/lib scp hadoop-2.2-viprfs-1.0.1.jar [email protected]:/usr/lib/hadoop-hdfs/lib/ Repeat this for all nodes

Verify connectivity by running a hadoop list command hadoop fs –ls / hadoop fs -ls viprfs://ObjectBucket.ObjectTenant.HSK:9040/

Configure Pivotal HD HDFS integration with EMC ViPR

ViPR HDFS is a POSIX-like Hadoop compatible file system (HCFS) that enables you to run Hadoop 2.0 applications on top of your ViPR storage infrastructure. You can configure your Hadoop distribution to run against the built-in Hadoop file system against ViPR HDFS, or any combination of HDFS, ViPR HDFS, or other HCFSs available in your environment. When you run Hadoop applications on top of ViPR, the ViPR HDFS software manages both the metadata and the data access so the Hadoop NameNode and Hadoop data nodes do not perform those functions.

The core-site.xml contains configuration information about HDFS. For Hadoop to use ViPR for HDFS requires configuration of the core-site.xml file. The latest release of ViPR integrates data services API with Hadoop, by instantiating a java client jar file on each worker node as a Hadoop Compatible File System (HCFS). The jar file is placed in the hadoop classpath on each node.

Modifying the core-site.xml to add the ViPRFS java classes, defining the ViPRFS URI, allows Hadoop cluster’s to point to the ViPR HDFS implementation. The MapReduce jobs can then read and write data directly on the ViPRFS file system.

For the configuration examble delow the fs.defaultFS is viprfs://ObjectBucket.ObjectTenant.HSK

In a  previous  post I showed how to setup object data services within ViPR.  Building on that example we would create a tenant called "ObjectTenant" that has access to the object data store. We would then create a bucket from the ViPR service catalog called "ObjectBucket" that supplies both s3 and HDFS access to the object data store (Example of bucket creation here). Finally "HSK" in the defaultFS name is used per ViPR instance. Meaning that a single core-site.xml could be configured to access mutliple ViPR data services instances.

The Table below shows the configuration that need to be place into the core-site.xml file

Property	Description
<property> <name>fs.defaultFS</name> <value>viprfs://ObjectBucket.ObjectTenant.HSK</value> </property>	File system default name ObjectBucket = The name of the bucket ObjectTenant = The tenant name HSK = FS installation name defined in the next property
<property> <name>fs.vipr.installations</name> <value>HSK</value> </property>	Add this custom ViPR HDFS property as a comma-separated list of names. The names are further defined by the fs.vipr.installation. [installation_name].hosts property to uniquely identify sets of ViPR data nodes. The names are used as a component of the authority section of the ViPR HFDS file system URI.
<property> <name>fs.vipr.installation.HSK.hosts</name> <value>10.10.81.160</value> </property	Add this custom ViPR HDFS property specifying the IP addresses of the ViPR cluster's data nodes or the load balancers for each name listed in the fs.vipr.installations property. Specify the value in the form of a comma-separated list of IP addresses.ViPR data services IP address. Only other value that needs to be changed is “HSK”, which is defined in the previous property and the Data Services IP address. Ip address can also be FQDN.
<property> <name>fs.viprfs.impl</name> <value>com.emc.hadoop.fs.vipr.ViPRFileSystem</value> </property>	Custom ViPR file system property. No changes necessary
<property> <name>fs.AbstractFileSystem.viprfs.impl</name> <value>com.emc.hadoop.fs.vipr.ViPRAbstractFileSystem</value> </property	Custom ViPR file system property. No changes necessary
<property> <name>fs.permissions.umask-mode</name> <value>000</value> </property>	Custom ViPR file system property. No changes necessary
<property> <name>fs.viprfs.auth.anonymous_translation</name> <value>CURRENT_USER</value> </property>	Custom ViPR file system property. No changes necessary

 ViPR custom Core-site.xml properties and values

Hadoop distribution	ViPR HDFS JAR
Pivotal	hadoop-2.0.x-alpha-viprfs-1.0.1.jar
Cloudera	hadoop-2.0.x-alpha-viprfs-1.0.1.jar
Pivotal HD	hadoop-2.2.viprfs-1.0.1.jar

ViPR Java client version

This section will show you how to use VIM to edit the core-site.xml. During the install of PHD you have the option of editing the core-site.xml. If you didn’t edit it you can use VIM of another editor to add the EMC ViPR values to the file. Do the edits to the core-site.xml to all nodes in your cluster.

cd /etc/gphd/conf.gphd-2.0.1/ vim core-site.xml

Using the values from table above edit the fs.defaultFS property: <name>fs.defaultFS</name> <value>viprfs://ObjectBucket.ObjectTenant.HSK</value>

Add the remaining properties and values from table 4 to the end of the file. Quit and save. Your file should look like the screen shot below:

Upload the client jar file to the phcc server. Copy it to all the nodes in your cluster using the scp command. Do this for all hosts. scp hadoop-2.0.x-alpha-viprfs-1.0.1.jar [email protected]:/usr/lib/gphd/hadoop/lib

Validate that the connection is working with a hadoop list command. First using the ViPR URL then just a normal Hadoop list: hadoop fs -ls viprfs://ObjectBucket.ObjectTenant.HSK:9040/ hadoop fs –ls /

EMC ViPR Hadoop Starter Kit

Today EMC launched the Hadoop starter kit (HSK) ViPR edtion. These kits are designed to help deploy a Hadoop enviornment and use EMC ViPR as a hadoop compatiable file system for HDFS. There are 3 seperate guides that each focus in on how to deploy ViPR data services, create an S3/HDFS object bucket, and integrate the leaders in commercial hadoop distributuions, Pivotal, Cloudera, and Horton works. The starter kit can be found here:

https://community.emc.com/docs/DOC-34442

 

Below are some demo videos of ViPR's HDFS integration with Pivotal, Cloudera, and Hortonworks

EMC ViPR and PivotalHD Integration Demo from James Ruddy on Vimeo.

 

 

EMC ViPR and Cloudera Integration Demo from James Ruddy on Vimeo.

EMC ViPR and Hortonworks Integration Demo from James Ruddy on Vimeo.

How to build a Data Lake Part 2 –The Architecture

 

In my previous post I shared the origins of the Data Lake pilot within the EMC Open Innovations Lab. Based off that criteria we decided we needed to build an new analytics environment that would allow for real time data processing and the ability to compare it to historical data. Add into the mix that data will be ingested in 1 second increments at a large volumes.

The picture above shows Pivotal HAD Architecture that is shown in most presentations and blog posts about Pivotal. There are 4 main products. Pivotal Hadoop, which supplies Yarn, Zookeeper, HDFS (in our pilot this is provided by Isilon), Hbase, Pig, Hive, Manhout, Map Reduce (v1 for legacy, v2 is provided by Yarn), and virtual extensions. Hawq for SQL, which resides on its own server. Hawq segments get installed in our pilot on every node that has a Hadoop Yarn role. Command Center which uses icm to deploy all the software in the architecture and manages the whole environment. GemFireXD for in memory data processing with persistent data located in HDFS. Finally at the bottem of the lake there is Isilon which runs the namenode tracker and data node for HDFS.

Spring gets it own VM where we write adapters to ingest data. We will not be using Sqoop, Data Loader, Flume or Unified storage services.

As we designed the architecture the decision was made to feature functionality over performance for phase 1. During phase 2 of this project that will look at the performance of this analytics environment which includes the use of Data Scientists to optimize the processing of the data. For our servers we decided that virtualization would be our way to go. I’m a big believer in the virtualize everything strategy and with Hadoop I have yet to be shown that there is any downside. Also by virtualizing the environment we can get the added bonus of elasticity. Here is a great wiki from apache on virtualizing hadoop (http://wiki.apache.org/hadoop/Virtual%20Hadoop)

As noted in the wiki, virtualizing HDFS has some downsides. To address this we have decided to use EMC Isilon for HDFS. Isilon supports the HDFS interfaces for the DataNode and NameNode to host data and metadata that creates a shared storage model allowing for muti-tenacy including the use of multiple hadoop clusters and HDFS application access to the same data set. Accessing HDFS, for hadoop or any other application is as simple as pointing the HDFS clients to the DNS name of the Isilon cluster. Using an Isilon shared storage model for Hadoop has many benefits. When using traditional HDFS on DAS the Storage to compute ratio is fixed. Add more compute means adding more storage. By separating the 2 we can elastically scale our compute, using VMware, and our storage with Isilon independently.

For real time analytics we went with GemFireXD. GemFireXD is an in memory data grid that allows to real time analytics with the ability to persist the data to HDFS. Data will be ingested into the grid using adapters created in spring XD. GemFireXD allow data to be kept in memory for a predefined amount of time. When that time is reached it is flushed to HDFS using the DFS of the namenode. We will use a 4 node grid with each VM having 120GB of memory. While in memory the data is also avaliable to be queried by Hawq.

For Hadoop we went with Pivotal Hadoop (PHD). All hadoop distributions are based on the same code set from Apache. Since in memory data processing was a must, the integration between GemFireXD and Hawq was key. Hawq adds the ability to use SQL queries against data that resides in memory (Gem) and persistent (HDFS). The PHD cluster will have 18 nodes running the following roles:

Role	Server
PCC	PCC
Name/Data Node	Isilon2
Yarn Resource Manager/History Server	PHDnode1
Node Manager	PHDnode3-16
Zookeeper	PHDnode3,5,7
Hive	PHDnode1
Hive Metastore	PHDNode2
Hbase Master	PHDNode2
Region Server	Phdnode3-16
Hawq Master	Hawq
Hawq Segment	PHDnode3-16

Hawq allows sql queries to be performed on the data. It is installed on to its own node and has integration into Hadoop and HDFS through segments that are installed on the yarn nodes. Using the Pivotal Framework Extender (PFX) SQL quieries can also be perfomed on data in th eGemFireXD cluster.

So what does this all look like? Here is a viso of the environment.

Note that we are still deciding the best way to ingest data, and haven’t fully decided on the analytic applications to use yet. Ill discuss this more in future posts. Also threads between map jobs and HDFS have to be tuned so that is not set in stone as we will tune this during testing.

And in the virtual environment it looks like this:

The next blogs will show how to install and integrate all of these components.

How to build a Data Lake Part 1 –The Pilot

So I’ve seen a lot of blogs recently talking about the Data Lake. What it is and what it means. My favorite has been Steve Todd's which gives a good high level over of what a data lake is. In the EMC open innovations lab (OIL) we are constantly working with Internal EMC BU’s, different parts of the federation (RSA, VMware, Pivotal), and customers to explore and pilot new technologies. Recently we began working with a customer who had a unique challenge. Their old legacy analytic systems and databases could not help them with their manufacturing needs. In real time they want to be able to run analytics on data coming off of the assembly line while also comparing it to historical data. This is the perfect use case for a data lake! As I prepped for this pilot I found lots of blogs that talked about the theory and the “why” you should build a data lake. I also found lots of internal EMC/Pivotal docs about how to install different piece parts, but no overarching documentation on how to build a data lake. As always this blog focuses on the how. This Pilot is just kicking off and as it proceeds I will be updating this blog with the architecture, how to deploy and configure it, load data, and run analytic applications. Should be a fun project!

Openstack Icehouse Cinder and EMC ViPR

Last week EMC released their newest storage plugin for Icehouse Cinder. This version FOUND HERE now supports Havana, Icehouse, and EMC ViPR 1.1. I have previously blogged on how to setup the ViPR driver for version 1.0 and Havana (HERE). The plugin also now supports multiple backends simultaneously. The instillation of the newest version of the plugin has one major change, and that is the use of the ViPR CLI client to create volumes on ViPR virtual storage pools. This blog will show you how to setup the new driver

Enviornment

3 nodes installed with CentOS and RDO Icehouse

ViPR is backed by 2 storage systems. Isilon for NFS and VNX for iSCSI Block

The Physical arrays have been abstracted into Virtual array’s.

From the Virtual array we have created Virtual pools. This blog we will be using the Tier0-SSD Virtual pool to create cinder volumes.

We have alos created a project called ObjectProject. 

Install Cinder Driver

To install the cinder driver we first have to install the ViPR CLI on the Cinder node.

mkdir cli

cd cli

Download the CLI from your VIPR vm using the wget command

wget http://10.10.81.161:9998/cli

Extract the installer with the Tar command

tar -xf cli

Install the ViPR CLI

./Installer_viprcli.linux

The ViPR cli install begins. Note the instillation directory, Port number, and the ViPR hostname, which must be a FQDN.

Set the path environment using the source command

source /opt/vipr/cli/viprcli.profile

With the CLI installed we can begin the driver installation

cd /tmp

wget github.com/emcvipr/controller-openstack-cinder/archive/master.zip

Unzip the driver

unzip master

Copy the ViPR subdirectory into the cinder volume directory

cd controller-openstack-cinder-master/cinder/volume/drivers/emc/

cp -avr vipr /usr/lib/python2.6/site-packages/cinder/volume/drivers/emc/

Edit the cinder.conf file

vim /etc/cinder/cinder.conf

At the top of the file in the default section enter the following

volume_driver = cinder.volume.drivers.emc.vipr.emc_vipr_iscsi.EMCViPRISCSIDriver

vipr_hostname=vipr.ebc.emc.local

vipr_port=4443

vipr_username=root

vipr_password=password

vipr_cli_path=/opt/vipr/cli/bin

vipr_tenant=Provider Tenant

vipr_project=ObjectProject

vipr_varray=CinderiSCSI

vipr_cookiedir=/usr/cookie

rpc_response_timeout=300

Enter the correct information for your environment. Note that the default tenant in ViPR is “Provider Tenant”

Restart the cinder service

service openstack-cinder-volume restart

Create a volume type

cinder --os-username admin --os-tenant-name admin type-create Tier0

Map the volume type to a virual pool in ViPR

cinder --os-username admin --os-tenant-name admin type-key Tiero set ViPR:VPOOL=Tier0-SSD

Note type key can be the ID number or name.

On the ViPR GUI we add the cinder host. Click Add

  

Enter the host information. Click Save

The host is added to ViPR

Under the network tab select the netwok associated with the array. In this example it’s the ISCSI-NET network

You will see the 2 IP ports from the array already assigned to this network. Click the Add button and select Add Host Ports

Scroll through the list till you find the iqn of the cinder host. Select it and click Add

The Port is added to the network. Click Save

On the Openstack Horizon dashboard we can now create a volume. Give the volume a name and select the type created earlier. Click Create Volume

On the ViPR GUI you can see the volume being created by viewing the resource tab under users.

We can also view the creation of the volume in Unisphere.

We now have a ViPR iSCSI block volume in Cinder for use.

If you have any issues with LUN creation check the /var/log/cinder/volume.log file. Details of volume creation are logged there.

Install Configure and Use s3fs with the EMC ViPR s3 API

In my lab I am working on a springXD project and I needed a linux host to mount a ViPR s3 bucket. Google led me to the s3fs project. S3fs is a FUSE-based file system backed by Amazon S3 or s3 compatable storage. It allows a linux host to mount a bucket as a local file system. This blog will show how to install s3fs and configure an EMC ViPR s3 bucket for use. This install is done with CentOS 6.4 and ViPR 1.1. You can see my previous post on how to set up object servies for ViPR HERE 

On the CentOS Host install the following packages with YUM:

yum install gcc libstdc++-devel gcc-c++ curl curl* curl-devel libxml2 libxml2* libxml2-devel openssl-devel mailcap

After the YUM install we install FUSE with the following commands:

NOTE: FUSE version is latest as of March 5, 2014

cd /usr/local/src

wget http://sourceforge.net/projects/fuse/files/latest/fuse-2.9.3.tar.gz

tar -xzvf fuse-2.9.3.tar.gz

mv fuse-2.9.3 fuse

cd fuse/

./configure --prefix=/usr

make

make install

export PKG_CONFIG_PATH=/usr/lib/pkgconfig:/usr/lib64/pkgconfig/

ldconfig

modprobe fuse

pkg-config --modversion fuse

Next install s3fs with the following commands:

NOTE: s3fs version is latest as of March 5, 2014

cd /usr/local/src

wget https://s3fs.googlecode.com/files/s3fs-1.74.tar.gz

tar -xzvf s3fs-1.74.tar.gz

mv s3fs-1.74 s3fs

cd s3fs

./configure --prefix=/usr

make

make install

From the emc community page https://community.emc.com/docs/DOC-27823 we can find the networking guide for ViPR Data services. For applications to access ViPR s3 buckets we must create DNS or local host records. First we need an “A” record for our data services VM that was deployed to host ViPR data services. For this blog we have the following

Object.cto.emc.local                          10.10.81.160

Next we need to create a 2^nd “A” record for the bucket. In this case my bucket will be called buckets3 so the DNS record is

Buckets3.object.cto.emc.local         10.10.81.160

Note that the bucket IP address is the same as the data services IP address. Every bucket we create that we want an s3 application to have access to will need an associated “A” record created.

Next we need to create a Base URL in ViPR and a bucket. The base url is the FQDN of the data services VM. It is used by application to prepend the base URL with the bucket name.

Under Data Services, Base URL, create a URL with the FQDN of the ViPR data services VM

Next we need to create a bucket. From the user menu select service catalog and click on Data Services

Select Create Bucket for Data Services

Give the bucket a name. From the drop down menus choose a virtual pool, project, object access and HDFS access. Click Save Note : In this example my bucket will have both object and S3 for access.

Back on the CentOS box we can finish the s3fs setup by creating a password file

vim /etc/passwd-s3fs

Enter the following into the file:

root:HzoWC4x2/bSt+CySRdkeqMbrajjHZPZyoKg4OQn8

Above is the AccessKeyID:SecretAccessKey The Secret key can be accessed from ViPR and is shown below.

chmod 640 /etc/passwd-s3fs

 

Retrieve the Secret Data store key from the ViPR web page under root->Manage data store Keys

Now we can mount the s3 bucket

The command to mount the bucket is: s3fs -o host=http://object.cto.emc.local:9020 buckets3 /mnt/s3 Note that the –o option points to the Base url and needs the ViPR data service port 9020 for s3 HTTP A df –h shows the s3fs mount

Configure Object Services for EMC ViPR 1.1

This blog is an update to my previous post on configuring object services for ViPR found HERE. That post was for ViPR 1.0. This post is for ViPR 1.1 and includes new configurations of Object Buckets and HDFS. The deployment for 1.1 is the same as 1.0 and can be found HERE

On the web client, under Data Services-> setup select the IP network to use. Use the same network that you have previously assigned to your file services.

Click Virtual Pools. Click Add

Create an Object Virtual pool. Give the virtual pool a name and select the type. Note this pool can support 3 different access types. Click Save

We now have a data services virtual pool that supports both Object (s3 and swift) and HDFS access.

Click Data Stores. Click Add

Give the data store a name. Use the drop down menu’s to select the Virtual array, File storage pool, and Virtual pool. An object file system will be created with in the virtual file pool. Choose the size of this file system. Click Save

The data store is created. Notice the mount point. You can verify its creation on your file array using putty or file system explore. All objects placed on the object store will go into this directory.

Click Tenant configuration. Give a tenant Namespace and use the drop downs to select the virtual pool and default project. Click Save.      A data pool is now setup on ViPR that can support HDFS, S3, and Swift access.

Install and Configure Openstack Neutron

 This blog will show you how to install Openstack Neutron using packstack on nodes running in a virtual environment. Configuration of the service is done at the command line. There is a lot of information avaliable on how to create networks and routers using the CLI, this blog will show you how to create networks and routers using the Horizon web page.

For my enviroment I am using Havanna RDO installed on top of CentOs 6.4. I have 3 nodes that are running as virtual machines in a VMware environment. I have also tested this with the IceHouse release and the install and configuration is valid for that release as well.

Nova nodes.

First thing to do is set the Promiscuous mode to accept on the vSwitches

All the nodes have 3 network interfaces. Eth0 is used for both the internal and external networking and Eth1/2 is used to connect to my iSCSI storage network. I use packstack with an answer file to install all 3 nodes. Below are the specific settings in the answer file for Neutron. CONFIG_NEUTRON_SERVER_HOST=10.10.81.169 CONFIG_NEUTRON_KS_PW=password CONFIG_NEUTRON_DB_PW=password CONFIG_NEUTRON_L3_HOSTS=10.10.81.169,10.10.81.168,10.10.81.167 CONFIG_NEUTRON_L3_EXT_BRIDGE=br-ex CONFIG_NEUTRON_DHCP_HOSTS=10.10.81.169,10.10.81.168,10.10.81.167 CONFIG_NEUTRON_LBAAS_HOSTS= CONFIG_NEUTRON_L2_PLUGIN=openvswitch CONFIG_NEUTRON_METADATA_HOSTS=10.10.81.169,10.10.81.168,10.10.81.167 CONFIG_NEUTRON_METADATA_PW=password CONFIG_NEUTRON_LB_TENANT_NETWORK_TYPE=gre CONFIG_NEUTRON_LB_VLAN_RANGES= CONFIG_NEUTRON_LB_INTERFACE_MAPPINGS=eth0:br-int CONFIG_NEUTRON_OVS_TENANT_NETWORK_TYPE=gre CONFIG_NEUTRON_OVS_VLAN_RANGES= CONFIG_NEUTRON_OVS_BRIDGE_MAPPINGS=eth0:br-ex CONFIG_NEUTRON_OVS_BRIDGE_IFACES=br-ex:eth0 CONFIG_NEUTRON_OVS_TUNNEL_RANGES=1:1000 CONFIG_NEUTRON_OVS_TUNNEL_IF = Install with the following command: packstack –answer-file=my_answers.txt

After the install completes we can view our hosts from the horizon web page verifying that all hosts were installed

We have to configure the br-ex interface. Use vim to edit the ifcfg-br-ex file

We use the setting shown above. Note that we use the ip information from eth0 here. We will replace the ifcfg-eth0 file with new settings in the next steps.

Delete the ifcfg-eth0 file. Use vim to create a new one.

The settings above will map eth0 to the newly created br-ex.

Next we change the veth settings in the ini file located in the /etc/neutron directory. You can copy and paste the lines in one shot below: for i in /etc/neutron/*.ini do sed -i "s/^[# ]*ovs_use_veth.*$/ovs_use_veth = True/g" $i done

Next we enable he metadata in the dhcp_agent.ini file.Copy and past the lines below in one shot: sed -i \ -e "s/^[# ]*enable_isolated_metadata.*$/enable_isolated_metadata = True/g" \ -e "s/^[# ]*enable_metadata_network.*$/enable_metadata_network = True/g" \ /etc/neutron/dhcp_agent.ini

NOTE: The last 6 steps must be repeated on all nodes that will be configured to use Neutron. In this case all 3 of my nodes need this configuration. Use the correct IP address for each node in the br-ex file.

Test that the the new br-ex and eth0 settings work by restarting the network service network restart After the configuration is complete and tested on all nodes, reboot the hosts.

Allow ICPM and SSH connections for the Default security group with the following commands: source /root/keystonerc_admin nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0 nova secgroup-add-rule default tcp 22 22 0.0.0.0/0

From the Horizon web page under Admin, Networks click “Create Network”

Create an internal network called Internal. Assign it to the admin project. Click “Create Network” Back on the Admin, Network page you will see the network created. Click “Create Network” again

Create an external network called External. Assign it to the admin project and use the check box to mark it as an External Network. Click “Create Network”.

We now have 2 networks created. We need to create subnets for them. To this click on the network name.

Click on the Create Subnet button

Above are the settings for my external network. This is the same network address scheme as the IP’s used for the br-ex interfaces. Click the Subnet Detail tab

Enable DHCP and create an allocation pool of IP’s that are available on the network. Note that these IPS will be used as Floating IP’s and at least one will be assigned to the external interface of the router will will create to allow the internal network to have access to the external network. Click “Create Subnet” when complete.

We can see the subnet is created and associated with the External network. Go back to the Admin, Network page and click the Internal network.

Click the “Create Subnet” button

Above are the settings for my internal network. The gateway IP will be used for the internal interface of the router. Click the Subnet Detail tab

Enable DHCP and create an IP allocation pool that will be given to instances attached to the Internal Network. Click “Create Subnet”

The Internal Subnet is created.

On the Project, Router page click “Create Router”

Give the router a name. Click “Create Router”

Set the gateway for the router by clicking “Set Gateway”.

Use the drop down to select the External Network we created called External. Click “Set Gateway”

The router is now created and connected to the External network. Click on the name “Router”

We can see the interfaces connected to the router and the IPS assigned. The status reads down until the first traffic flow through the interface.

Click the “Add Interface” button

Use the drop down to connect the internal network to the interface. Give it an IP address. Click “Add Interface”

The router now has 2 interfaces connecting it to both the external and internal network.

Click on the Projects, Network Topology page and we can see a visual representation of our network.

To test the network I deploy 3 instances. Nova will load balance so that every nodes runs one instance. I connect them to the internal network. The topology map show how they are connected and the IPS of each instance. Click on one of the instances and open a console.

From that instance we can ping the other 2 instances and the internal router interface

We can also ping a node on our external network. This verifies that Neutron is configured and working correctly.

EMC ViPR- Block and Object Storage for Openstack DEMO

This blog features a Demo of Using EMC ViPR for all your storage needs in an Openstack environment. Block storage for Openstack Cinder using iscsi and Object based storage for Glance, all through one management interface. Couple of things to note as you watch the demo. 1st the real coolness about Object based storage in Openstack is its ability to take a storage bucket and make it accessible to multiple API's like Swift, S3, and Atmos. So you can load data however you choose and its then available to all these object interfaces. Besides offering Block and Object, ViPR can also offer NFS services. So if you interested in supporting Live Migration features in KVM, you can set these services up dynamically (the video Doesn’t show this part). Finally, in my demo I only have one block storage array, but ViPR has the ability to setup virtual pools that span multiple array’s.

Here is the Demo:

EMC Vipr- Block and Object Storage for Openstack from James Ruddy on Vimeo.