This is a continuing series on how to build a data lake. Welcome to part7
Part 6
Over the past couple of weeks Ive been blogging on how to create a data lake. These blogs included the architecture and how to install of a Data Lake with PivotalHD, GemfireXD, and Hawq all running as VM's within a VMware infrastructure using EMC Isilon for HDFS.
The final step is to create an application and run some analytics. This demo is based off the retail demo that can be found on github:
https://github.com/Pivotal-Field-Engineering/retail-demo-xd
What you will see in the demo is an application in SpringXD that uses MAdLib, built with Redis that shows data flow from ingestion, tapping data to the in memory data grid GemfireXD, with all data ending up in Isilon HDFS that can be queried using Hawq PXF.
A big thanks to Louis Mugano for help with the application build.
Data Lake Demo with Pivotal and Isilon from James Ruddy on Vimeo.
Comments