Spark Acceleration

Apache Spark is becoming the first choice platform for organizations the develop new Big Data and Machine Learning applications. Spark has many attractive properties such as support for popular programming languages, and a unifying approach to different styles of analytics. But high-performance and scale is a key benefit of Spark due to its fast in-memory operations across the Spark cluster.

Read the Bigstream whitepaper to understand how Hyper-acceleration provides the world’s fastest Spark runtime

How can Bigstream claim the “World’s Fastest Spark?” It is because we can deliver Hyper-acceleration of Spark using both software and hardware accelerators in a single platform, all without touching your Spark code. Bigstream’s advanced compiler technology and runtime provides the best software acceleration, and seamless acceleration using hardware accelerators such as FPGAs and GPUs.

Bigstream software acceleration

  • Native compilation - Bigstream performs runtime generation of C++ code from the dataflow representation from Spark, which is then natively compiled
  • Vectorization - Data parallel computations in Spark are vectorized for more efficient processing on multi-core CPUs or FPGAs
  • Custom Data Connectors - Accelerated native access to Apache Kafka, Amazon S3, and Hadoop FileSystem (HDFS)
  • High-Speed Data/Document Parsers for JSON, CSV, Parquet, and Avro formats

Bigstream Hardware Acceleration

  • Pre-compiled FPGA bit files automatically program with FPGA without special skills or tools.
  • Intelligent slicing of Spark dataflows in the Bigstream runtime routes computations to the optimum processor, providing additive acceleration (CPU + FPGA)
  • Bigstream hypervisor integrates natively with processing back-end to ensure zero changes to application code

Some examples of Hyper-accelerated data pipelines with Apache Spark:

Spark-powered Data Warehouse

Spark Predictive Analytics

Spark Machine Learning

Read the Bigstream Benchmark Report to see how Bigstream Hyper-acceleration will transform your Big Data security analytics.