In my last blog post I showed how to use an existing puppet module on the puppet forge to deploy Cloudera hadoop. After deploying the nodes the next thing to do is deploy and configure all the roles for the nodes and get your cluster up and running. The rest of this blog will show you how to start the Cloudera express wizard and deploy your first Cloudera hadoop cluster.
|
First, here is my final site.pp file ( see this blog post for puppet deployment) that I used to deploy a 4 node hadoop cluster. I deployed using parcels as opposed to RPM’s because this allows me to use Cloudera manager to automate configuration and deployment of a managed hadoop cluster and use it for updating packages. |
|
|
Here is the puppet.conf file that I used on all hosts. The only change was the cert name to correspond to the actual host. |
|
|
After the puppet run completes successfully log on to Cloudera manager with a web browser. Format is http://fqdn:7180 Username = admin Password = admin |
|
|
The express wizard launches. Enter your licensing info or use the 60 day trial and click Continue |
|
|
After entering the licensing information you need to restart the management server service. On the management server run this command service cloudera-scm-server restart |
|
|
After the service restarts you need to login again |
|
|
Cloudera then thanks you for using its product. Click Next |
|
|
You can search for your hosts that you want to use for the cluster. Select them and click Continue |
|
|
Choose your method, and versions to install. Click Continue Note: Parcels gives you the most funcinality, including automating deployment and rolling updates. |
|
|
Provide the ssh credentials for the hosts. Click contnue |
|
|
Deployment of the appropriate packages to hosts begins. When complete click Continue |
|
|
Selected parcels are installed to the hosts. When completed click continue |
|
|
Cluster inspection runs and verifies the cluster. Click next when complete. |
|
|
Choose the services to deploy. You can always add more service later. |
|
|
You can inspect the roles and assign them to the hosts by clicking the “Inspect Role Assignment”. Again you can change this later. I went with the defaults. |
|
|
Choose the database (external or Custom) that you want to use. I used the embedded that was setup by my puppet manifest. |
|
|
Test the connection. When verified click continue. |
|
|
You can edit the setting for the roles. I took the default. Click continue |
|
|
The roles are assigned and the services are started. Click continue when complete |
|
|
Cloudera congratulates you for a successful install. Click Continue |
|
|
The manager starts. You can view the details of the services and roles, and edit them as appropriate. Notice on my setup there is a configuration warning on hdfs1. The warning is shown below: This cloudera blog talks about this setting http://blog.cloudera.com/blog/2009/02/the-small-files-problem/ The cluster is now operational and ready for use! |
Comments