“Big data” - hardware-accelerated compression for Apache Hadoop

October 29, 2013 // By Graham Prophet
Exar's AltraHD is the first hardware-accelerated compression solution for Apache Hadoop, enabling all applications in the Hadoop stack to be transparently compressed using hardware acceleration, delivering performance and storage efficiency; AltraHD maximises performance, Exar says, while minimising the storage footprint of Hadoop Clusters.

AltraHD combines Exar's software technology with its hardware compression accelerators to remove I/O bottlenecks and optimise the storage capacity for Hadoop. AltraHD integrates seamlessly into the Hadoop stack, and is able to transparently compress all files, including files that are stored using the file system as well as files stored locally as intermediate data outside of the file system. AltraHD is the only compression solution for Hadoop to offer the following key features:

- Application transparent file system filter that sits below the Hadoop Distributed File System (HDFS) to compress all files that are using HDFS.

- Compression codec that compresses intermediate data during the MapReduce phase of Hadoop processing.

- Exar's high performance PCIe-based hardware compression card that automatically accelerates all compression operations, maximising performance while offloading the host CPU. A single card provides up to 3.2 Gigabyte/sec of compression throughput.

"Exar has taken big data analytics to the next level," stated Rob Reiner, Exar's director of marketing for Data Compression and Security products. "AltraHD typically improves Terasort performance by 50 - 100%, and typically provides a 3x - 5x savings in storage capacity, enabling unmatched Hadoop cluster performance and efficiency, and establishing AltraHD as the industry leading solution."

Exar; www.exar.com