Spark of The Future with DataBricks

Big Data Beard

Spark is one of the most popular platforms for Data Science and AI development today and the team at DataBricks is driving massive innovation in this community, both open-source and commercially.

Cory sat down with Reynold Xin, Co-Founder and Chief Architect, and Michael Armburst, Principal Architect, at DataBricks to understand the history of Spark development and uncover the massive developments in the platform available now and in the near future making Spark more accessible and easier for Data Scientists and Engineers alike.

This podcast was recorded at Spark+AI Summit in San Francisco, California in April 2019.

Links

DataBricks

Reynold on LinkedIn & Twitter

Michael on LinkedIn & Twitter

DeltaLake and Announcement

Apple Presentation on DeltaLake

MLFlow

Videos from Spark+AI Summit 2019

Reynold Keynote Recording on Koalas

Michael Keynote Recording on Delta

Did you know we are on Spotify? Tune in to The Big Data Beard!

Don’t forget that we are going to Domino Data Labs’ upcoming REV2 Conference in NYC May 23-24. Visit the REV Conference Website and use promo code BDB_REV100 to save $100 on your pass!

Music from this episode is by Andrew Belle. Please go check him out...you'll thank us!