Iceberg at Netflix and Beyond with Ryan Blue
Podcast: Play in new window | Download
Subscribe: RSS
Apache Iceberg is an open source high-performance format for huge data tables. Iceberg enables the use of SQL tables for big data, while making it possible for engines like Spark and Hive to safely work with the same tables, at the same time.
Iceberg was started at Netflix by Ryan Blue and Dan Weeks, and was open-sourced and donated to the Apache Software Foundation in November 2018. It has now been adopted at many other companies including Airbnb, Apple, and Lyft.
Ryan Blue joins the podcast to describe the origins of Iceberg, how it works, the problems it solves, collaborating with Apple and others to open-source it, and more.
This episode is hosted by Lee Atchison. Lee Atchison is a software architect, author, and thought leader on cloud computing and application modernization. His best-selling book, Architecting for Scale (O’Reilly Media), is an essential resource for technical teams looking to maintain high availability and manage risk in their cloud environments.
Lee is the host of his podcast, Modern Digital Business, an engaging and informative podcast produced for people looking to build and grow their digital business with the help of modern applications and processes developed for today’s fast-moving business environment. Listen at mdb.fm. Follow Lee at softwarearchitectureinsights.com, and see all his content at leeatchison.com.
Sponsorship inquiries: sponsor@softwareengineeringdaily.com
Sponsors
This episode of Software Engineering Daily is brought to you by Starburst. Struggling to deliver analytics at the speed your users want without your costs snowballing?
For data engineers who battle to build and scale high quality data pipelines, Starburst’s data lakehouse platform helps you deliver exceptional user experiences at peta-byte scale, without compromising on performance or cost.
Trusted by the teams at Comcast, Doordash, and MIT, Starburst delivers the adaptability and flexibility a lakehouse ecosystem promises on an open architecture that supports – Apache Iceberg, Delta Lake and Hudi, so you always maintain ownership of your data. Want to see Starburst in action? Get started today with a free trial at starburst.io/sed.
You’ve tried every Kanban and collaboration tool to keep your team in sync love you find the one that’s United everyone to propel your projects forward. While being intuitive to use and graphically pleasing. You can either approach Miro from the kind of team you’re on engineering or product management, for example, or by use case maybe technical diagramming or scale product planning. I particularly like the built in support for product development like estimation and retrospectives, take a look at what 60 million users already know that Miro is a secure and visual workspace for innovation. Find simplicity in your most complex projects with Miro. Your first three mural boards are FREE when you sign up today at miro.com/podcast.
WorkOS is a modern identity platform built for B2B SaaS. It provides seamless APIs for authentication, user identity, and complex enterprise features like SSO and SCIM provisioning. It’s a drop-in replacement for Auth0 (auth-zero) and supports up to 1 million monthly active users for free.
It’s perfect for B2B SaaS companies frustrated with high costs, opaque pricing, and lack of enterprise capabilities supported by legacy auth vendors. The APIs are flexible and easy to use, designed to provide an effortless experience from your first user all the way to your largest enterprise customer.
Today, hundreds of high-growth scale-ups are already powered by WorkOS, including ones you probably know, like Vercel, Webflow, and Loom. Check out workos.com/SED to learn more.
