DoK Talks#103 -Performant and Version-Aware Analytics With Spark & lakeFS on K8s // Itai Admi

Data on Kubernetes Community

Player FM - Internet Radio Done Right

27 subscribers

הוסף לפני four שנים

תוכן מסופק על ידי Data on Kubernetes Community. כל תוכן הפודקאסטים כולל פרקים, גרפיקה ותיאורי פודקאסטים מועלים ומסופקים ישירות על ידי Data on Kubernetes Community או שותף פלטפורמת הפודקאסט שלהם. אם אתה מאמין שמישהו משתמש ביצירה שלך המוגנת בזכויות יוצרים ללא רשותך, אתה יכול לעקוב אחר התהליך המתואר כאן https://he.player.fm/legal.

We Have The Receipts

1
Battle Camp S1: Reality Rivalries with Dana Moon & QT 1:00:36

לפני 19 ימים1:00:36

הפעל מאוחר יותר

רשימות

לייק

אהבתי

1:00:36

Do you have fond childhood memories of summer camp? For a chance at $250,000, campers must compete in a series of summer camp-themed challenges to prove that they are unbeatable, unhateable, and unbreakable. Host Chris Burns is joined by the multi-talented comedian Dana Moon to recap the first five episodes of season one of Battle Camp . Plus, Quori-Tyler (aka QT) joins the podcast to dish on the camp gossip, team dynamics, and the Watson to her Sherlock Holmes. Leave us a voice message at www.speakpipe.com/WeHaveTheReceipts Text us at (929) 487-3621 DM Chris @FatCarrieBradshaw on Instagram Follow We Have The Receipts wherever you listen, so you never miss an episode. Listen to more from Netflix Podcasts.…

לפני 3 years 39:25

M4A•בית הפרקים

https://go.dok.community/slack
https://dok.community/
ABSTRACT OF THE TALK
Spark and lakeFS are revolutionizing large scale data processing that is version-aware. Is it possible to run this architecture over Kubernetes? We’ll cover the fastest way to get this environment up and running, and the benefits you get with it. Finally we’ll show how horizontal scaling and the lakeFS Hadoop Filesystem avoid processing bottlenecks as workloads increase.
BIO
Itai is a R&D team leader at Treeverse, the company behind open-source lakeFS. He thrives on finding creative solutions for complex problems, especially if it involves code. Previously, Itai worked at Microsoft and Ridge on data infrastructure, tooling, and performance. Itai received his B.Sc degree in Computer Science and an MBA from Tel Aviv University.
KEY TAKE-AWAYS FROM THE TALK
- Importance of building reproducible data pipelines.
- Managing your data the same way you're managing your code.

243 פרקים

#Tech #Demetrios

Data on Kubernetes Community

DoK Talks#103 -Performant and Version-Aware Analytics With Spark & lakeFS on K8s // Itai Admi

Data on Kubernetes Community

27 subscribers

published לפני 3 years

שתפו

M4A•בית הפרקים

243 פרקים

#Tech #Demetrios

Tất cả các tập

Data on Kubernetes Community

1
Implementing Data & Databases on K8s within the Dutch Government | DoKC Town Hall 44:54

לפני 1 year44:54

44:54

Implementing Data & Databases on K8s within the Dutch Government Presented by Sebastiaan Mannem , Director at Mannem Solutions A small walkthrough of projects within the Dutch government running databases on OpenShift. This talk shares success stories, provides a proven recipe to `get it done,` and debunks some of the FUD. Related Links DoKC Website - https://dok.community/ DoKC Meetups - https://www.meetup.com/data-on-kubernetes-community/ Join Slack - https://join.slack.com/t/dokcommunity/shared_invite/zt-1vgv7ymz7-YtLFvZicrcLP9fS3o_r2_w…

Data on Kubernetes Community

1
Unsticking Ourselves from Glue: Migrating PayIt’s Data Pipelines to Argo Workflows and Hera | DoKC Town Hall 23:17

לפני 1 year23:17

23:17

Unsticking Ourselves from Glue: Migrating PayIt’s Data Pipelines to Argo Workflows and Hera Presented by Matt Menzenski, Senior Software Engineering Manager, Payitgov At PayIt, we’ve been deploying applications to Kubernetes almost since the beginning of the company. Our data workloads, however, have run instead in AWS Glue. This has worked well enough for the reporting use cases that have been the main focus of this team historically. However, at the beginning of 2022, the PayIt data team began building out a new data platform, and in the process, ran into a number of challenges with Glue. In this talk, I will share the difficulties that we encountered with building, deploying, and orchestrating ETL pipelines in AWS Glue, our decision process for moving those workloads into Kubernetes, and the ELT architecture that we’ve arrived at today. Related Links DoKC Website - https://dok.community/ DoKC Meetups - https://www.meetup.com/data-on-kubernetes-community/ Join Slack - https://join.slack.com/t/dokcommunity/shared_invite/zt-1vgv7ymz7-YtLFvZicrcLP9fS3o_r2_w…

Data on Kubernetes Community

1
Repel Boarders! How to find a Kubernetes operator that really protects your data | DoKC Town Hall 19:22

לפני 1 year19:22

19:22

Repel Boarders! How to find a Kubernetes operator that really protects your data Presented by Robert Hodges, Altinity Operators are a godsend for managing data in Kubernetes. But how about protecting it? We'll explore security threats to cloud native databases and show what protection you should look for in operators. Finally we'll introduce a new Data on Kubernetes Community project to develop security standards for database operators in Kubernetes. Related Links DoKC Website - https://dok.community/ DoKC Meetups - https://www.meetup.com/data-on-kubernetes-community/ Join Slack - https://join.slack.com/t/dokcommunity/shared_invite/zt-1vgv7ymz7-YtLFvZicrcLP9fS3o_r2_w…

Data on Kubernetes Community

1
DoK + Apache Spark | DoKC Town Hall 19:52

לפני 1 year19:52

19:52

DoK + Apache Spark Presented by Holden Karau, Spark Committer and Open Source Engineer at Netflix In this brief talk, Holden will cover some of the best practices from trying to deploy both small and large scale Spark on Kube. Related Links DoKC Website - https://dok.community/ DoKC Meetups - https://www.meetup.com/data-on-kubernetes-community/ Join Slack - https://join.slack.com/t/dokcommunity/shared_invite/zt-1vgv7ymz7-YtLFvZicrcLP9fS3o_r2_w…

Data on Kubernetes Community

1
DoK @ Comcast - Deliver Business Outcomes & Improved DevX with Data Services on K8s | DoKC Town Hall 16:43

לפני 1 year16:43

16:43

DoK @ Comcast: Delivering Business Outcomes & Improved DevX with Data Services Running on Kubernetes Presented by Greg Otto, Executor Director, DevX Platforms & Charles Ju, Principal Engineer Transforming how to deliver measurable value using data on Kubernetes, while providing psychological safety. If you just sighed, you’re one of the many people leaders and engineers who know the challenges of making these changes. In this talk, we will share our transformation journey, the “Months to Minutes” outcomes we achieved, the architecture approach, and the human journey from one of our engineers. Related Links DoKC Website - https://dok.community/ DoKC Meetups - https://www.meetup.com/data-on-kubernetes-community/ Join Slack - https://join.slack.com/t/dokcommunity/shared_invite/zt-1vgv7ymz7-YtLFvZicrcLP9fS3o_r2_w…

Data on Kubernetes Community

1
DoK Talks - What is Kafka? The rise of one of the world's most used streaming data technologies // Abbey Russell 15:28

לפני 2 years15:28

15:28

Abbey Russell, PM at Cockroach Labs, shared the backstory on how and why Kafka was created. Along the way, you'll learn about - Who Franz Kafka was - Kafka's earliest use at Linkedin in 2010 - Why organizations like Uber/Coursera/Mailchimp use it today - Future of Data Streaming To find out more about how organizations are benefitting from running streaming workloads on Kubernetes, check out our 2022 research report- https://dok.community/data-on-kubernetes-2022-report/ https://dokcommunity.slack.com/ https://dok.community/ About Data on Kubernetes Community (DoKC) DoKC is an independent tech community where practitioners go to run data workloads on Kubernetes. It is managed by Constantia.io. We produce a range of resources for end users looking to run DoK including: - DoK Landscape An interactive landscape of products and services for running data workloads on Kubernetes. (https://dok.community/landscape/) - Videos and Blogs Hundreds of videos and blogs including how tos, tutorials, use cases, case studies, and more. (https://dok.community/landscape/) (https://dok.community/blog/) - DoK Reports DoKC works with an independent research firm to annually produce the industry’s only DoK-focused report. (https://dok.community/dok-reports/) - Livestreams and Local Meetups Weekly livestreams on DoK-related topics including demos, tutorials, visionary talks, panels, and more. (https://www.meetup.com/data-on-kubern...) - DoK Days @ KubeCon We host hybrid events twice a year for people to get together and learn in-person or online. Details are published on our main website. (https://dok.community/) Connect With Us: https://dok.community/ https://twitter.com/DoKcommunity https://www.linkedin.com/company/6667... https://github.com/dokc…

Data on Kubernetes Community

1
DoK Talks - (almost)Everything you need to know about stateful cloud native network applications // W Watson 43:39

לפני 2 years43:39

43:39

https://go.dok.community/slack https://dok.community/ https://youtu.be/KjiK6eXYO34 DoK Talk with W Watson, Founder at Vulk Co-op

Data on Kubernetes Community

1
The Outer Nerd #001 - Dungeons & Dragons - Why should you care? // Abhi Vaidyanatha, Fabian Met & Chase Christensen 58:25

לפני 2 years58:25

58:25

https://dokcommunity.slack.com/ https://dok.community/ ABSTRACT OF THE TALK Fabian, Chris and Abhi will discuss their passion for roleplaying games, and what they can teach us about the power of community, improvisation, and using our creativity.

Data on Kubernetes Community

1
DoK Talks #155 - Databases at the edge with K3s and ARM devices // Sergio Méndez 49:40

לפני 2 years49:40

49:40

https://go.dok.community/slack https://dok.community/ https://youtu.be/KjiK6eXYO34 ABSTRACT OF THE TALK In this talk Sergio is going to present different ways to store data at the edge using different databases and Long Horn as a storage class. All this running on a Raspberry Pi and showing and small application using a database running at the edge. BIO Sergio Méndez is a systems engineer and professor of operating systems at USAC Guatemala university. His work at the university is related to teaching and researching cloud native technologies with his students. He has experience working on DevOps, and MLOps using open source technologies at work. About open source communities, he is involved in the CNCF Community, promoting students into the CNCF Ecosystem and hosting a Cloud Native meetup in Guatemala. He has been a speaker at several conferences such as KubeCon, WTFisCloudNative, and Kubernetes Community Days. He is also a Linkerd Ambassador and author of Edge Computing systems with Kubernetes with Packt publisher. KEY TAKE-AWAYS FROM THE TALK How to storage data at the edge using databases…

Data on Kubernetes Community

1
DoK Talks #154 - StatefulSets in K8 // Srinivas Karnati 31:55

לפני 2 years31:55

31:55

https://go.dok.community/slack https://dok.community/ Link: https://youtu.be/n_thXwyJNSU ABSTRACT OF THE TALK Deploying Stateless applications is easy but this is not the case for Stateful applications. StatefulSets are the K8s API object that helps to manage stateful application. Learn about what Stateful sets are, how to create, How it differs from Deployments. BIO Passionate about Cloud Native, Kubernetes, Developer relations KEY TAKE-AWAYS FROM THE TALK This talk is focused on basics of StatefulSet, how StatefulSet differs from Deployments, How to manage Stateful app using StatefulSet…

Data on Kubernetes Community

1
Data-driven Diversity, Equity, and Inclusion // Lisa-Marie Namphy, Melissa Logan, Tiffany Jachja, Audra Montenegro & Cortney Nickerson (DoK Day North America 2022) 19:50

לפני 3 years19:50

19:50

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY )

Data on Kubernetes Community

1
Formula 1 telemetry processing using Apache Kafka on Kubernetes // Paolo Patierno (DoK Day North America 2022) 15:36

לפני 3 years15:36

15:36

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY ) Video - https://youtu.be/4cPVRWOK-_E ABSTRACT Apache Kafka is the de facto data streaming platform used for ingesting vast amounts of data and processing them in real-time. Low latency analytics are vital if users are to react to events as fast as possible and to effectively shape future decision making. Together with Kubernetes, it allows to develop cloud oriented analytics solutions which are highly scalable. During this session, we will explore an Apache Kafka based solution running on Kubernetes to help a Formula 1 team gain insights during a race. The “real” data will come from a well-known racing game and will be processed by our application in real-time, making us feel like real track-side F1 engineers!…

Data on Kubernetes Community

1
Choosing Kubernetes for Stateful Applications // Akshay Ram & Peter Schuurman (DoK Day North America 2022) 18:31

לפני 3 years18:31

18:31

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY ) Video - https://youtu.be/Y4tdy9lctEI ABSTRACT Learn how customers are increasingly deploying stateful applications on Kubernetes to benefit from portability, economies of scale, and built-in orchestration capabilities. This talk will include how customers choose between using Kuberentes, or a data Software as a Service (SaaS) and stateful capabilities of Kubernetes across two dimensions - the application orchestration and the storage layer. Also learn about MariaDB SKYSQL, a database software as a service that runs thousands of StatefulSet Pods across multiple zones and regions on Kubernetes.…

Data on Kubernetes Community

1
Kubernetes 360º - Data driven observability - from Secrets to logs // Ben Hirschberg (DoK Day North America 2022) 17:11

לפני 3 years17:11

17:11

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY ) Video - https://youtu.be/A1ch4AhKoeQ ABSTRACT If there’s one thing that everyone can agree on - it’s that the sheer scale and complexity of Kubernetes operations is growing constantly. What’s more, cloud native environments are becoming more and more expensive to operate and manage, as well as increasingly difficult to secure. On the bright side, there is a growing ecosystem of exceptional open source tools to help overcome this complexity, and provide greater situational awareness to what’s happening in your many and multiple Kubernetes clusters. In this talk, we’ll take a deep dive on how to understand everything that is happening with stateful workloads in your Kubernetes clusters from health to security - including the management and monitoring of your clusters, troubleshooting and debugging, through full security scanning in multiple data perspectives- from configMaps, Secrets, hostPaths and many more security driven checks for proper role-based access control, vulnerabilities and misconfigurations. We will demonstrate how you can have a 360º-view of your Kubernetes clusters in a single open source dashboard leveraging tools like Lens and Kubescape, and ultimately enforce policies with Open Policy Agent.…

Data on Kubernetes Community

1
Shifting Left Stateful Applications In Kubernetes // Viktor Farcic (DoK Day North America 2022) 15:52

לפני 3 years15:52

15:52

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY ) Video - https://youtu.be/LymPjH6HA3E ABSTRACT Stateless apps are easy to manage. More often than not, a Kubernetes Deployment, with a Service, Ingress, and Horizontal Pod Autoscaler (HPA) is enough. Almost everyone can do it. But, when it comes to stateful applications, things become a bit more complicated. We might need a database and storage. We might need to manage database users and schema. We might need to consider quite a few other things. Stateful apps are harder for everyone, especially if we want to shift left and enable developers to do it themselves. In this talk, we'll try to make the management of stateful applications easy for everyone. We'll accomplish that by creating easy-to-consume services that are made specifically for the needs of our organizations. We'll see how to create new Kubernetes Custom Resource Definitions (CRDs) and controllers using Crossplane. Those controllers will envelop all the tools, resources, and processes we might need. As a result, ops can focus on creating such services while everyone else can consume them (create and manage everything related to stateful apps) without opening JIRA tickets and waiting for others to complete their tasks.…

Data on Kubernetes Community

1
Medical - Healthcare Data on Kubernetes // Olyvia Rakshit & Prasad Dorbala (DoK Day North America 2022) 13:41

לפני 3 years13:41

13:41

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY ) ABSTRACT Healthcare organizations are transforming their applications and embracing digital platforms for efficient patient care. Today, compute at the edge, plays a critical role in deploying innovative healthcare applications that promise new approaches to patient care. Connected care platforms at scale are turning to Kubernetes infrastructure for deployment of telehealth and telemedicine apps. These kinds of applications often need real time image processing and inferencing at the edge, which in turn, require a distributed application platform on Kubernetes. They also need secure foundations for a regulatory compliant framework for wider adoption. To elaborate further, a distributed platform or a multi-cluster infrastructure is necessary for healthcare applications. All medical application platforms are data intense. For eg. Medical applications that need to stream video data, i.e move large quantities of data need secure connections from edge to cloud. Smart healthcare applications are also seeing the value of multi-cloud deployments. The ability to choose different technologies like AI, data storage systems or some specific relational database systems is driving multi-cloud adoption in healthcare solutions. A multi-cluster connectivity fabric is thus needed for data intensive healthcare applications to distribute across geographical boundaries. The fabric needs to have inbuilt automation and the smarts to make the best use of compute resources in a distributed infrastructure, for improved patient healthcare.…

Data on Kubernetes Community

1
Highly Available Postgres Clusters In Kubernetes // John Long & Jonathan Gonzalez (DoK Day North America 2022) 15:04

לפני 3 years15:04

15:04

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY ) ABSTRACT A practical session about running Highly Available PostgreSQL in Kubernetes. The primary objective will be to demonstrate how to set up a reliable architecture in a Kubernetes cluster to achieve low RTO and RPO. This will be covered by going over the various Kubernetes native components of our fully open source operator CloudNativePG[0]. It will be demonstrated how these components allow for a simple adoption of PostgreSQL into Kubernetes environments. If time and technical constraints permit there will be a live demo at the end of the presentation; if for some reason that is not possible there will be a link to a recorded demo. https://cloudnative-pg.io/…

Data on Kubernetes Community

1
Inter-Cluster PostreSQL on Kubernetes // Julian Fischer (DoK Day North America 2022) 17:07

לפני 3 years17:07

17:07

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY ) ABSTRACT In this talk you’ll explore how to run a PostgreSQL cluster across multiple Kubernetes clusters. Learn what challenges arise when using asynchronous streaming replication in a set of Kubernetes clusters spanning across several geographical regions. It will be discussed how such a topology can be established using Kubernetes and Kubernetes Add-Ons. You will learn which qualities the system has, where such a distributed system plays its strengths and where it may be problematic to use it. At the end of the talk you will have an increased understanding about how to create multi-regional data service instances using PostgreSQL as an example.…

Data on Kubernetes Community

1
Open Source Databases on Kubernetes- Best Practices // Peter Zaitsev (DoK Day North America 2022) 16:04

לפני 3 years16:04

16:04

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY ) ABSTRACT So you’re looking to run your Open Source Database on Kubernetes. What best practices should you follow and what pitfalls should you avoid ? In this presentation we will look at how to run stateful applications on Kubernetes overall as well as what is particularly important for databases - we will cover high availability, security, backups and disaster recovery. Finally we will show how these practices can be implemented with Percona Operators for MySQL, MongoDB, PostgreSQL - one of the leading solutions to run Open Source Databases on Kubernetes…

Data on Kubernetes Community

1
The Kubernetes Native Database // Jeffrey Carpenter (DoK Day North America 2022) 16:26

לפני 3 years16:26

16:26

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY ) ABSTRACT In the software industry we’re fond of terms that define major trends, like “cloud native”, “Kubernetes native” and “serverless”. As more and more organizations move stateful workloads to Kubernetes, we’ve started to see these terms applied to data infrastructure, where they can get overtaken by marketing hype unless we work to define them. In this talk, we’ll examine two different databases, TiDB and Apache Cassandra, in order to identify what it means for a database to be Kubernetes native and why it matters. We’ll look at points including: - The differences between cloud native, Kubernetes native, and serverless - How databases become Kubernetes native - Benefits of Kubernetes native databases - How Kubernetes can better support databases…

Data on Kubernetes Community

1
Databases on Kubernetes: Why are they important? // With Bhavin Shah, Xing Yang, Gabriele Bartolini & Patrick McFadin (DoK Day North America 2022) 34:51

לפני 3 years34:51

34:51

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY ) ABSTRACT Kubernetes has crossed the chasm, but what about stateful applications and databases? Join us for this panel discussion and learn more about how organizations are deploying different databases like PostgreSQL and Cassandra on Kubernetes, what are the benefits of running databases on Kubernetes, and how the ecosystem is working towards making these things boring, so you can focus on your applications! We will have an interactive discussion with the hosts and guests of the Kubernetes Bytes podcast, and open it up to the audience to ask questions and learn more about the what, why, and how about Databases on Kubernetes!…

Data on Kubernetes Community

1
Data streaming on Kubernetes // Yaniv Ben Hemo (DoK Day North America 2022) 13:51

לפני 3 years13:51

13:51

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY ) ABSTRACT I will cover what is the current data streaming on k8s landscape, why it is important, use cases, and what are the challenges needed to solve

Data on Kubernetes Community

1
Architecting Your First Event Driven Serverless Streaming Applications on K8 // Timothy Spann (DoK Day North America 2022) 13:29

לפני 3 years13:29

13:29

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY ) ABSTRACT Once you have built a topic in Apache Pulsar, you will quickly see the need to build event-driven applications. This can require a lot of decisions on what framework to use, where to run it, how to deploy it, and how to manage these applications on Kubernetes cloud natively. I will walk you through step-by-step in building Pulsar Functions which is the easy way to design, test, develop, integrate, deploy, monitor, and manage serverless streaming applications in Java and Python. Together we will build a full application as an Apache Pulsar function and enjoy the power of running it in the cloud for IoT events and add any routing, transformation, or machine learning that we need to accomplish our business requirements. Through FunctionMesh we run on Kubernetes natively. In this talk, you will deploy ML functions to transform real-time data on Kubernetes.…

Data on Kubernetes Community

1
Fybrik - A Kubernetes based platform for governed data use // Flora Gilboa-Solomon, Alexey Roytman, Maryna Strelchuk & Barry Hijkoop (DoK Day North America 2022) 20:59

לפני 3 years20:59

20:59

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY ) ABSTRACT Data is the foundation for business value. However, in many enterprises, it is spread across different data stores, public/private clouds, and on-premises. The use of data is governed by regulatory requirements and enterprise policies and enterprises face dynamic data residency challenges. Fybrik ( https://fybrik.io ) is an open-source, Kubernetes-based infrastructure that enables easier access to data distributed across hybrid and multi-cloud environments, while enforcing data governance policy according to the business’s needs. In this talk, we will share use cases from a pilot project carried out with ING (a multinational banking corporation). Fybrik enabled ING the use of their data across geographical borders, across clusters in a secure, automated, and compliant manner.…

Data on Kubernetes Community

1
The Challenges of Data Processing On Kubernetes - A look at Spark, Flink, Dask, and Ray // Holden Karau (DoK Day North America 2022) 20:09

לפני 3 years20:09

20:09

From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY) ABSTRACT This talk will go through both the improvements that have been made in Kubernetes for batch analytic workloads as well as some of the current pain experienced by users and developers moving their workloads to Kube. In this talk you will learn about how we “cheated” back in the YARN and Mesos days to make things go fast, why Kubernetes doesn’t like those cheats, and what some alternatives are.…

Data on Kubernetes Community

1
Scaling our SaaS offering to thousands of clusters // Dax McDonald (DoK Day North America 2022) 21:04

לפני 3 years21:04

21:04

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY ) ABSTRACT Sourcegraph is a code intelligence platform that helps our customers to understand their code better. As we have scaled up, we are starting to run hundreds of instances for our customers in separate kubernetes clusters. Running dozens of distinct clusters with a stateful application that scales horizontally and vertically has been challenging. Sourcegraph's inherently stateful nature requires thoughtful planning to achieve resilience while making tradeoffs to keep costs down. This talk will discuss our methodologies to allow our cluster to scale while minimizing drift across multiple customer deployments. I will also cover our use of Terraform, ArgoCD, and Helm to manage the application and infrastructure. Finally, this talk will discuss our disaster recovery strategies that allowed our organization to achieve SOCII compliance while iterating quickly. If you are looking to scale out a managed offering at your organization, learn from our mistakes and come to this talk!…

Data on Kubernetes Community

1
Why we decided to migrate our Jaeger storage to ClickHouse on Kubernetes // Arul Jegadish Francis (DoK Day North America 2022) 13:48

לפני 3 years13:48

13:48

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY ) Abstract We at OpsVerse provide a DevOps tools platform with fully-managed open source-based tools. One of our key offerings is a holistic observability platform. Metrics and logs are straightforward to aggregate, however traces – which are collected using CNCF Jaeger – were left with some holes in advanced insights. In this talk, I will explain why we decided to switch our Jaeger storage backend to ClickHouse, and why we decided to run it on Kubernetes. I will also share tips and tricks to migrate and run ClickHouse on Kubernetes.…

Data on Kubernetes Community

1
Building a Digital Factory for the Sheet Metal Industry // Elie Assi (From the DoK Day North America 2022) 20:48

לפני 3 years20:48

20:48

From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY) Abstract We develop systems to digitize the sheet metal industry with the belief that they should cooperate with each other in an open way. We are convinced that the future lies in creating a software ecosystem that interconnects all levels of the company and even manages to communicate with supplier and customer systems, making for more agile management throughout the entire value chain. One of our newer projects is providing a multi-tenant cloud service to our clients. The chosen database for the service we are developing is MongoDB. After testing with docker containers on virtual machines, we decided to finally host the service in Kubernetes because we wanted something to be able to scale at large for our clients being a service that will be consumed by a multitude of our clients, so to the database in order to reduce latency.…

Data on Kubernetes Community

1
How we built our Big Data Stack (almost) entirely on top of Kubernetes // Neylson Crepalde (From DoK Day NA 2022) 16:00

לפני 3 years16:00

16:00

From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY) Abstract Working with Terabytes of data is a major challenge for organizations both in terms of architecture and cost. In recent years, a new paradigm has emerged in the world of Big Data, that is, implementing the entire architecture for processing massive data from a microservices approach. This architecture proved to be scalable, efficient, with excellent performance, failsafe, portable between different cloud providers and at a lower cost than conventional architectures. In this talk, I will present how my team managed to implement this architecture based on Kubernetes and what were the main gains we had with the innovation.…

Data on Kubernetes Community

1
Dok Talks #153 - CRD Panel // Eyar Zilberman & Álvaro Hernández 58:05

לפני 3 years58:05

58:05

https://go.dok.community/slack https://dok.community We are going to speak about CRDs, and discuss considering them as higher level entities that we normally consider them. CRDs normally are kind of a byproduct of an operator. But in reality, they can be considered as the user-facing API of the operator surface. And as such, we would like to introduce conversations about even standardizing some of them, or creating "specifications" out of some CRDs. Take this for example: there are some CRDs that could be, potentially, common across multiple operators. Think of a CRD to represent an S3 (or any other storage object) bucket. For operators that store data in object stores, each and every one of them are creating their own versions. But shouldn't we have some "standardized" `ObjectStorage` CRD, without a controller, that all could use? We'll be joined by John Sanda of DataStax who works on the K8ssandra project, Alvaro Hernandez who is the CEO of Ongres and Postgres superhero, and Eyar Zilberman Co-Founder and Chief Product Officer at Datree.…

ברוכים הבאים אל Player FM!

Player FM סורק את האינטרנט עבור פודקאסטים באיכות גבוהה בשבילכם כדי שתהנו מהם כרגע. זה יישום הפודקאסט הטוב ביותר והוא עובד על אנדרואיד, iPhone ואינטרנט. הירשמו לסנכרון מנויים במכשירים שונים.

תקשיבו ל-500+ נושאים

27 subscribers

דומה לData on Kubernetes Community

Amazon eGift Card - Amazon Logo (Animated)

USANOOKS Microfiber Cleaning Cloth Grey - 12 Pcs (12.5"x12.5") - High Performance - 1200 Washes, Ultra Absorbent Microfiber Towel Weave Grime & Liquid for Streak-Free Mirror Shine - Car Washing Cloth

Ailun 3 Pack Screen Protector for iPhone 16 Pro Max [6.9 inch] + 3 Pack Camera Lens Protector with Installation Frame,Sensor Protection,Dynamic Island Compatible,Case Friendly Tempered Glass Film

פודקאסטים ששווה להאזין

Data on Kubernetes Community « » DoK Talks#103 -Performant and Version-Aware Analytics With Spark & lakeFS on K8s // Itai Admi

DoK Talks#103 -Performant and Version-Aware Analytics With Spark & lakeFS on K8s // Itai Admi

פודקאסטים ששווה להאזין

ברוכים הבאים אל Player FM!

The Let Them Theory: A Life-Changing Tool That Millions of People Can't Stop Talking About

The Let Them Theory: A Life-Changing Tool That Millions of People Can't Stop Talking About

Mighty Patch™ Original patch from Hero Cosmetics - Hydrocolloid Acne Pimple Patch for Covering Zits and Blemishes in Face and Skin, Vegan-friendly and Not Tested on Animals (36 Count)

Bounty Paper Towels Quick Size, White, 16 Family Rolls = 40 Regular Rolls (Packaging May Vary)

The Let Them Theory: A Life-Changing Tool That Millions of People Can't Stop Talking About

דומה לData on Kubernetes Community

מדריך עזר מהיר

Data on Kubernetes Community « »
DoK Talks#103 -Performant and Version-Aware Analytics With Spark & lakeFS on K8s // Itai Admi