I began working at the EMC Open Innovations lab (OIL) in July, and our first project was to work with the Isilon business unit to help support Big Data initiatives. We started with 2 projects, Deploying Splunk on Isilon reference architecture ( SPLUNK) and the EMC Hadoop starter kit (HSK)! The HSK utilizes VMware big data extension (BDE) to automate deployment of all the major hadoop distributions (PivotalHD, Apache, Cloudera, Hortonworks) in a VMware environment. It then show how to use EMC Isilon storage for native HDFS storage. The point of the guide was to enable someone (like myself), who had no experience with hadoop or big data, to rapidly deploy hadoop environments using already existing equipment. If you have a VMware environment, and you own and Isilon array, you can deploy HSK in about an hour. The guide is meant to help a real world IT issue, Shadow IT. Since most infrastructure guys and gals are not programmers, there is a significant ramp up time for IT to deploy hadoop clusters. This includes learning, testing, and procurement of “commodity” hardware (which requires a budget). What we’ve seen in the field is that many hadoop projects were starting in Amazon as IT was seen as a road block to beginning Hadoop projects.
Here is a demo of the HSK:
And the download link:
Love your work!
Posted by: Ned Shawa | 06/16/2014 at 03:28 AM