bigstream blog

Bigstream Adds Computer Architecture Specialist John Davis to its Leadership Team

Acceleration visionary  joins Bigstream as Chief Product Officerhs_back_000

June 15, 2017

Bigstream welcomes microarchitecture expert and acceleration visionary John Davis to the team as our Chief Product Officer. John has been our most valuable advisor in helping us successfully deliver the first hyper-acceleration platform to the cloud. Prior to joining us, John was part of the founding team for the FlashBlade product at Pure Storage, where he built and led the hardware group from concept to GA design. He has also done research and systems design at Sun Microsystems and Microsoft Research.

John has published over 30 refereed conference and journal papers in Computer Architecture, Systems, and Bioinformatics. He also holds over 30 issued or pending patents (see below).

Commenting on the John’s arrival, Bigstream CEO Maysam Lavasani said:

“We are very happy to welcome John Davis to Bigstream. John brings incredible energy and clarity of thought to everything he does. As an advisor, John has already contributed greatly to our product vision and company direction. As a full time part of our team, I expect his impact will be far greater”

As a further introduction to John, we sat down for an interview recently and this is what he had to say.



Describe your role at Bigstream  

My main role is to help define the product by translating what customers need into products and related features, the Bigstream acceleration ecosystem.

How does Bigstream fit in the overall big data, machine learning and AI space?

The large cloud companies have demonstrated a way to exert extreme effort to accelerate their critical big data, ML and AI applications.  More and more small and medium size companies are being built on these fundamental algorithm, but without the performance advantages. Any company that wants to use these applications or frameworks can accelerate them with Bigstream’s technology, instantly and easily!

Who in the IT, development and business communities has the most to gain from Bigstream Hyper-acceleration?

All three benefit, IT becomes simpler and more efficient, developers can use high-level frameworks to increase productivity and Bigstream dramatically improves performance, and business becomes more agile by reducing time to intelligence as well as, creating more intelligence than could be done before.

As Chief Product Officer, how will you measure the success of the product and the company?

There are many ways to recognize success: enabling customer products that were not possible before, time saved in transforming data into intelligence, and making acceleration table stakes for business success. Acceleration is generating new opportunities to create data products and insights that were not possible before. This enables our customers to react faster in an agile global business environment.  This creates efficiency through acceleration by saving time, reducing energy and the physical footprint of the digital world. First to knowledge will be the differentiator between business success and failure. When customers cannot be competitive without our product, we know we are doing the right thing.

List of John Davis Patents

  • US Patent 7,290,116 : Level 2 Cache Index Hashing to Avoid Hot Spots, Greg F. Grohoski, Manish
    Shah, John D. Davis, Ashley Saulsbury, Cong Fu, Venkatesh Iyengar, Jenn-Yuan Tsai, and Jeff Gibson,
    October 2007
  • US Patent 7,444,499: Method and System for Trace Generation Using Memory Index Hashing, John D.
    Davis and Cong Fu, October 2008
  • US Patent 7,598,766: Customized Silicon Chips Produced Using Dynamically Configurable
    Polymorphic Network, Martha Mercaldi-Kim, Mark Oskin, John D. Davis, Todd Austin and Mojtaba
    Mehrara, October 2009
  • US Patent 8,037,437: Optimizing Systems-on-a-Chip Using the Dynamic Critical Path, John D. Davis,
    Mihai Budiu and Hari Kanna, October 2010
  • US Patent 8,103,847: Storage Virtual Containers, John D. Davis, and Vijayan Prabhakaran, Jan. 2012
    US Patent 8,131,660: Reconfigurable Hardware Accelerator for Boolean Satisfiability Solver, John
    Davis, Zhangxi Tan, Fang Yu, Lintao Zhang, March 2012
  • US Patent 8,688,954: Remapping of Inoperable Memory Blocks Using Pointers, John D. Davis, August
  • US Patent 8,868,825: Nonrepeating Identifiers in an Address Space of a Non-Volatile Solid-State
    Storage, John Hayes, Shantanu Gupta, John Davis, Brian Gold, Zhangxi Tan, July 2014
  • US Patent 8,874,835 B1:Data Placement Based on Data Properties in a Tiered Storage Device System,
    John Davis, Ethan Miller, Brian Gold, John Colgrove, Peter Vajgel, John Hayes, Alex Ho, Jan. 2014
  • US Patent 8,874,836 B1: Scheduling Policy for Queues in a Non-Volatile Solid-State Storage, John
    Hayes, Shantanu Gupta, John Davis, Brian Gold, Zhangxi Tan, July 2014
  • US Patent 8,904,209: Estimating and Managing Power Consumption of Computing Devices Using Power
    Models, John D. Davis, Moises Goldszmidt, and Suzanne M. Rivoire, November 2011
  • US Patent 8,972,649: Writing Memory Blocks Using Codewords, John D. Davis, Parikshit Gopalan,
    Mark Manasse, Karin Strauss, and Sergey Yekhanin, September 2012
  • US Patent 9,003,144 B1: Mechanism for Persisting Messages in a Storage System, John Hayes, Igor
    Ostrovsky, Robert Lee, Shantanu Gupta, Rusty Sears, John Davis, Brian Gold, June 2014
  • US Patent 9,082,512 B1: Die-Level Monitoring in a Storage Cluster, John D. Davis, John Hayes,
    Zhangxi Tan, Hari Kannan, Nenad Miladinovic, August 2014
  • US Patent 9,110,789 B1: Nonrepeating Identifiers in an Address Space of a Non-Volatile Solid-State
    Storage, John Hayes, Shantanu Gupta, John Davis, Brian Gold, Zhangxi Tan, October 2014
  • US Patent 9,201,600 B1: Storage Cluster, John Hayes, John Colgrove, John D. Davis, Jan. 2015
  • US Patent 9,213,485 B1: Storage System Architecture, John Hayes, John Colgrove, John D. Davis, Feb.
  • US Patent 9,357,010 B1: Storage System Architecture, John Hayes, John Colgrove, John D. Davis, Feb.
  • US Patent 9,367,243 B1: Saclable Non-Uniform Storage Sizes, John Hayes, John Colgrove, Robert Lee,
    Joshua Robinson, Peter Valgel, John Davis, Par Botes, June 2104
  • US Patent 9,437,943 B1:Stacked Symmetric Printed Circuit Boards, John D. Davis, March 2015
  • US Patent 9,477,554 B2: Mechanism for Persisting Messages in a Storage System, John Hayes, Igor
    Ostrovsky, Robert Lee, Shantanu Gupta, Rusty Sears, John Davis, Brian Gold, Dec 2015

Bigstream Benchmark Report Shows 54% Cost Savings on Spark Deployments

June benchmark results show a 2.89X speedup of Sparkspark-rocket

By George Demarest, Vice President of Marketing, Bigstream

June 13, 2017

Bigstream is in the acceleration business. Our approach to hyper-acceleration is at the platform level, as opposed to point solutions that require special APIs, custom coding, and changes to IT and DevOps processes. This means that as we add new platforms and features, the entire platform benefits. And Bigstream customers reap the benefits without ever changing a line of code.

This month’s Bigstream Benchmark Report shows a nearly 300% performance gain over the baseline open source Apache Spark 2.1 on the same configuration. This is the result of a complete performance overhaul of Spark using Bigstream Hyper-acceleration technology.  To get a better understanding of what Hyper-acceleration is, check out this interview with Bigstream CEO Maysam Lavasani.

Benchmarks are part of our daily processes and from time to time it makes sense to share performance data as we refine and expand our hyper-acceleration product.    The numbers and types of testing we do will evolve over time, but we use a combination of standard industry benchmarks  like TPC-DS (decision support), as well as some specific big data and machine learning use cases such as ETL, data ingest and parsing, SQL analytics and vertical industry use cases like AdTech real-time bidding, FinServ trading systems, and Retail analytics.

Here is a summary of this month’s results running Apache Spark with the Bigstream Hyper-acceleration Layer on Amazon EMR versus unaccelerated Apache Spark on the same configuration.

avro-tpc-ds-june csv-tpc-ds-june

Here is a look at the full results of the run:


All tests were done on standard Amazon EC2 server instances (m4.4xlarge) running on Intel Xeon processors.

For a full accounting of the results, take a look at the Bigstream Benchmark Report for July 2017.  In that report, you will see the following:

  • Test environment: Amazon EMR cluster of four m4.4xlarge instances, Apache Spark 2.1.1
  • Average acceleration: 2.89X using csv data, 2.47X using Avro data
  • Maximum acceleration:  3.86X on TPC-DS Query #9 (csv), 3.11X on TPC-DS Query #27 (Avro)
  • Average monthly cost savings on Amazon EMR: 54% (csv) and 46% (Avro)

Check out the report to get a bit more detail about the testing environment and how we worked out the cost savings. Things are going to get even more interesting when we publish our next benchmark report. In that report, we will include some testing of FPGA powered servers.


Bringing Hardware Acceleration to Data Science Without Touching Your Code

Part 1 of an interview with Maysam Lavasani, co-founder and CEO of Bigstream

On March 14 of this year, Bigstream announced the availability of the Bigstream Hyper-Acceleration Layer on Amazon EMR and for on-prem use. After years of research and almost 2 years of stealth development, the introduction of Bigstream hyper-acceleration for Apache Spark, Spark SQL/Dataframes and other big data technologies adds a powerful, frictionless technology to maximize the performance and economics of big data infrastructure.

But what is hyper-acceleration? What does it mean for data scientists, data engineers, DevOps teams and application architects? We sat down with Bigstream CEO Maysam Lavasani to get his perspective on hyper-acceleration, Apache Spark and the company that he co-founded. Read more »

The Bigstream story; presented and demonstrated

The Strata Conference was host to the first public demonstration of Bigstream Hyper-acceleration. See our announcement blog from the event.

Bigstream was honored with invitations to both the Solutions Showcase and the Startup Showcase.  We used that to demonstrate an AdTech use case that achieved 2.5X acceleration of  Spark SQL operations with no hardware acceleration and an ETL use cases that achieved about 10x acceleration using an FPGA.  Neither demos, nor benchmarks required changes to a single line of code. That is Hyper-acceleration.

Presented and demonstrated by Bigstream Solutions Architect, Roop Ganguly

Bigstream Announces the First Hyper-Acceleration Platform for Big Data on Amazon EMR

Bigstream achieves 2X-5X seamless acceleration of Apache Spark workloads, projects 10X-30X with GPUs/FPGAs

March 14, 2017
Strata + Hadoop World 2017, San Jose, CA

architectureToday, Bigstream announces the general availability of the industry’s only hyper-acceleration platform for big data. After years of research and nearly two years of stealth mode development, the Bigstream Hyper-acceleration Layer delivers 2X to 5X performance acceleration of Apache Spark workloads while requiring zero application code changes. Subsequent Bigstream releases will incorporate seamless acceleration using Field Programmable Gate Arrays (FPGAs) and Graphics Processing Units (GPUs) that can achieve up to 30X performance acceleration of Spark SQL, Machine Learning and other big data platforms.

Read more »

Bigstream to introduce hyper-acceleration for Apache Spark at Strata 2017

maysamThe following is a letter written by Bigstream Co-Founder and CEO,  Maysam Lavasani,  to our investors, advisors, and allies on Friday, March 10, 2017.


I am writing to let you know that Bigstream is about to announce our first generally available product. It is designed to deliver hyper-acceleration for a variety of big data and machine learning platforms, with Apache Spark being our initial focus. Following years of research and nearly two years of stealth mode development, Bigstream hyper-acceleration technology is going to change the way people think about acceleration using GPUs, FPGAs, and many-cores.

Read more »