PIPELINES SUCK.

 
 
vincent-burkhead-147113-unsplash.jpg
 
 

WE FEEL YOUR PAIN

We wrote our first MapReduce job in 2005. We’ve been deploying and scaling distributed systems like Hadoop, Cassandra, Spark, and Kafka since their infancy. We’ve learned a lot. This world needs a reset.

  • Data pipelines are too manual and too time-consuming. There’s lots of code, scheduling, trial and error, negotiations for infra resources, and a lot more code. Rinse and repeat for every new pipeline.

  • Pipelines break a lot. What ran fine last week is now taking twice the time, the data arrived late, or there’s an unexpected field. And now you’re digging through logs and tuning parameters to keep everything running.

  • There’s too much expertise required. Even if you live and breathe Airflow, you shouldn’t need to know the ins and outs of tuning instance sizing in AWS or the ideal number of Spark executors for every job. And if you’ve never worked with these tools… good luck.

 
 
 

THE BEST MINDS OF MY GENERATION ARE DELETING COMMAS FROM LOG FILES, AND THAT MAKES ME SAD.

- Michael Driscoll

Metamarkets

 

OPERATIONAL CREEP COMES FAST AND DISARMS DATA ENGINEERS FASTER THAN YOU CAN HIRE.

- Maxime Beauchemin

Downfall of the Data Engineer

 
 
 

DATA ENGINEERING RARELY BENEFITS FROM THE SANITY AND PEACE-OF-MIND DEVOPS PROVIDES MODERN ENGINEERS.

- Maxime Beauchemin

Downfall of the Data Engineer

 

COMPANIES UNDERESTIMATE THE EFFORT AND COST TO BUILD DATA PIPELINES…THEY’RE GENERALLY UNFINISHED PROJECTS.

- Abhishek Tiwari

Friends Don’t Let Friends Build Data Pipelines

 
 
Untitled+presentation.jpg

LUCKILY, THERE’S A BETTER WAY.

COMING THIS JULY.

 
 

we believe

  • Data comes first. Not pipelines. Nor streams. Nor, frankly, code. We believe data should be the language in which we communicate.

  • Declare, don’t implement. Across the technology landscape, declarative systems are quickly becoming the norm. This leads to less code, more stability, and a full night’s sleep.

  • Scale mandates automation. The complexity of interconnected systems grows exponentially. Manual construction requires people, and people don’t scale exponentially.

  • Agility + Iteration == Innovation. The network effect of data is real. Enabling more people to do more with data, faster, and safely is the defining characteristic of innovative teams.

 
 
 

DON’t like waiting?

 
 
 

Check out our Early Adopter Program.

* indicates required
 

Are you a member of the press? Let’s talk.

* indicates required