Hadoop is a great piece of technology and over the past year or so many innovative vendors have leveraged it to add value to their offerings. More recently, a new trend is emerging around Hadoop as  vendors are looking to combine their own product innovation with making the Hadoop platform easier to use. Early adopters have found the Hadoop stack to be complicated and often requires them to recruit new skill sets in order to fully leverage it.

Many vendors are moving up the stack, above the native MapReduce functions to create interfaces that work more easily with the platform.  HIVE, Chukwa and Pig are all being leveraged by vendors to foster easier use and integrated functionality.

I have been an open source fan for years so I’m especially interested in vendors who contribute back to the community. This morning SyncSort made an announcement that they will be contributing an external sort “plug-in” to the Hadoop project aimed at making the sort functionality more modular, flexible and extensible. The plug-in is designed to work with Syncsort’s DMExpress product but will also work with other solutions.

Syncsort has also announced the beta release of DMExpress Hadoop Edition (DHE). The new solution provides connectivity between DMExpress and the Hadoop Distributed File System (HDFS) giving the end user the opportunity to create jobs within the graphical interface of DMExpress where the job can then be passed to the MapReduce engine within Hadoop. DHE leverages the new sort plug-in to provide greater speed and functionality around sort processes. DHE also applies self-tuning features to insure better job performance and easier overall interaction with the Hadoop stack.

Hadoop is an important technology for companies looking to leverage their growing “big data” challenges and the innovation happening around the project will help to make it easier to adopt and integrate.