Databricks makes venture seem easy

by Pete SonsiniJun 30, 2014

What I love most about being an investor—well, there are two things. One is building relationships with amazing entrepreneurs, many of whom humble me daily with their brilliance and courage to change the world. The second is having a lens through which all the pieces of the vast tech landscape take shape to form one big picture, and the industry’s evolution seems as natural as it does novel.

Databricks, the big data analytics company behind Apache Spark that just raised a $33M round of financing led by NEA, embodies what I love about my job: I get to partner (again) with a dyed-in-the-wool entrepreneur, tackling one of the most critical barriers to unlocking the rich promise of big data.

I first partnered with Databricks founder Ion Stoica at Conviva, a pioneer in realtime processing to optimize delivery of Internet video at scale (an absolutely kick-ass company if you haven’t heard of them). It was one of my very first investments, and Ion was a co-founder. Several years later it was clear that a much broader opportunity was emerging around realtime processing of data sets; my investment in MapR really drove this home. Today, realtime processing is the linchpin for realizing the potential of big data. Nearly every new application we see must process in realtime to deliver value. Predictive analytics is pointless without realtime processing. If you can’t generate insights fast enough—such as serving offers to customers or identifying fraudulent transactions—the value of big data is utterly constrained.

Databricks has the goods. Its Spark platform, an open source general-purpose cluster computing framework for in-memory processing of massive data sets has emerged as the de-facto standard for real-time big data analytics. In fact, Spark has caught fire since its release and is currently one of the most active projects you will find–it has over 100 contributors, is included in each major distribution of Hadoop, and has over 100 organizations using it. We only expect these numbers to accelerate with the launch of Databricks’ new cloud platform, which brings the speed, ease of use, modularity and Hadoop compatibility to meet the developer community’s needs for realtime data processing and propel big data—and all its potential—into the mainstream.

This is an enormous opportunity, and it’s one I’ve been nurturing along the path for years. Truth be told, investing in Databricks felt as natural as breathing—and I am generally far from sanguine when it comes to new investments. Because if there is one thing that I don’t love about being an investor, it’s that losing money is in our job description—if you don’t lose any money, you aren’t taking enough risks.

I still find that unsettling, and I probably always will. Even after a productive, positive workday focused on things I can control, I’ll often spend a sleepless night worrying about the things I can’t. Because that’s our business—what you don’t know is just as important as what you do know, and careers can be made (and lost) overnight, from one deal. It’s become a litmus test for new investments: how much sleep am I likely to lose over this one?

But sometimes, you get lucky—because of a longstanding relationship with an incredible entrepreneur who you know is as legit as they come, or a clear view of how the future will unfold… or both. And in that moment, the venture business feels easy.

We are thrilled to lead the Series B investment in Databricks. While the decision to invest in this company couldn’t have been easier, we know the company will go through the twists, turns, ups and downs that every great company does. Of course no one knows what the future holds, but I’ve never felt better about a company’s prospects for changing the world of infrastructure software.