#22 DoK community: Vitess Operator for Kubernetes // Alkin Tezuysal

Data on Kubernetes Community

Player FM - Internet Radio Done Right

27 subscribers

הוסף לפני four שנים

תוכן מסופק על ידי Data on Kubernetes Community. כל תוכן הפודקאסטים כולל פרקים, גרפיקה ותיאורי פודקאסטים מועלים ומסופקים ישירות על ידי Data on Kubernetes Community או שותף פלטפורמת הפודקאסט שלהם. אם אתה מאמין שמישהו משתמש ביצירה שלך המוגנת בזכויות יוצרים ללא רשותך, אתה יכול לעקוב אחר התהליך המתואר כאן https://he.player.fm/legal.

Fixable

1
How to get your confidence back 41:05

לפני 21 weeks41:05

הפעל מאוחר יותר

רשימות

לייק

אהבתי

41:05

When negative feedback shakes your confidence, it can be difficult to get back to feeling like yourself at work. In this episode, Anne and Frances help a struggling listener who has spent years toning herself down in the workplace after being told that she was too assertive — now, she feels that her modest approach is holding her back. Together, they use Anne and Frances’s “trust triangle” framework to explore how empathy, authenticity, and logic can help you rebuild confidence and trust with your colleagues, and share helpful confidence hacks for getting comfy with discomfort. What problems are you dealing with at work? Text or call 234-FIXABLE or email fixable@ted.com to be featured on the show. For the full text transcript, visit ted.com/podcasts/fixable-transcripts Want to help shape TED’s shows going forward? Fill out our survey ! Hosted on Acast. See acast.com/privacy for more information.…

לפני 5 שנים 58:26

MP3•בית הפרקים

In this talk, I would like to uncover our newly announced Vitess Operator for Kubernetes. This talk demonstrates the sample implementation of Vitess in Kubernetes topology. I also explore common DBA tasks by demonstrating how they are handled in the Vitess ecosystem. Vitess, out of the box, comes with a lot of tools and utilities that one has to either incorporate or develop to manage MySQL topology. Let’s take a look at the capabilities of Vitess in these areas and demonstrate how they are performed under the operator realm.

Alkin Tezuysal has extensive experience in enterprise relational databases, working in various sectors for large corporations. With more than 20 years of industry experience, he has acquired skills for managing large projects from the ground up to production. For the past decade, he's been focused on e-commerce, SaaS, and MySQL technologies.

Alkin has managed and architected database topologies for high volume sites at eBay Intl. He has several years of experience in 24X7 support and operational tasks, as well as improving database systems for major companies. He has led global operations teams on Tier 1/2/3 support for MySQL customers.

In late 2020, he joined PlanetScale's expert engineering team over Vitess.

Open source database evangelist
Global database operations expert
Storyteller
Inspiring technical and strategic leader
Creative team builder
Speaker, mentor, and coach
Born to sail, forced to work

243 פרקים

#Tech #Demetrios

Data on Kubernetes Community

#22 DoK community: Vitess Operator for Kubernetes // Alkin Tezuysal

Data on Kubernetes Community

27 subscribers

published לפני 5 שנים

שתפו

MP3•בית הפרקים

In late 2020, he joined PlanetScale's expert engineering team over Vitess.

Open source database evangelist
Global database operations expert
Storyteller
Inspiring technical and strategic leader
Creative team builder
Speaker, mentor, and coach
Born to sail, forced to work

243 פרקים

#Tech #Demetrios

כל הפרקים

Data on Kubernetes Community

1
Implementing Data & Databases on K8s within the Dutch Government | DoKC Town Hall 44:54

לפני 1 year44:54

44:54

Implementing Data & Databases on K8s within the Dutch Government Presented by Sebastiaan Mannem , Director at Mannem Solutions A small walkthrough of projects within the Dutch government running databases on OpenShift. This talk shares success stories, provides a proven recipe to `get it done,` and debunks some of the FUD. Related Links DoKC Website - https://dok.community/ DoKC Meetups - https://www.meetup.com/data-on-kubernetes-community/ Join Slack - https://join.slack.com/t/dokcommunity/shared_invite/zt-1vgv7ymz7-YtLFvZicrcLP9fS3o_r2_w…

Data on Kubernetes Community

1
Unsticking Ourselves from Glue: Migrating PayIt’s Data Pipelines to Argo Workflows and Hera | DoKC Town Hall 23:17

לפני 1 year23:17

23:17

Unsticking Ourselves from Glue: Migrating PayIt’s Data Pipelines to Argo Workflows and Hera Presented by Matt Menzenski, Senior Software Engineering Manager, Payitgov At PayIt, we’ve been deploying applications to Kubernetes almost since the beginning of the company. Our data workloads, however, have run instead in AWS Glue. This has worked well enough for the reporting use cases that have been the main focus of this team historically. However, at the beginning of 2022, the PayIt data team began building out a new data platform, and in the process, ran into a number of challenges with Glue. In this talk, I will share the difficulties that we encountered with building, deploying, and orchestrating ETL pipelines in AWS Glue, our decision process for moving those workloads into Kubernetes, and the ELT architecture that we’ve arrived at today. Related Links DoKC Website - https://dok.community/ DoKC Meetups - https://www.meetup.com/data-on-kubernetes-community/ Join Slack - https://join.slack.com/t/dokcommunity/shared_invite/zt-1vgv7ymz7-YtLFvZicrcLP9fS3o_r2_w…

Data on Kubernetes Community

1
Repel Boarders! How to find a Kubernetes operator that really protects your data | DoKC Town Hall 19:22

לפני 1 year19:22

19:22

Repel Boarders! How to find a Kubernetes operator that really protects your data Presented by Robert Hodges, Altinity Operators are a godsend for managing data in Kubernetes. But how about protecting it? We'll explore security threats to cloud native databases and show what protection you should look for in operators. Finally we'll introduce a new Data on Kubernetes Community project to develop security standards for database operators in Kubernetes. Related Links DoKC Website - https://dok.community/ DoKC Meetups - https://www.meetup.com/data-on-kubernetes-community/ Join Slack - https://join.slack.com/t/dokcommunity/shared_invite/zt-1vgv7ymz7-YtLFvZicrcLP9fS3o_r2_w…

Data on Kubernetes Community

1
DoK + Apache Spark | DoKC Town Hall 19:52

לפני 1 year19:52

19:52

DoK + Apache Spark Presented by Holden Karau, Spark Committer and Open Source Engineer at Netflix In this brief talk, Holden will cover some of the best practices from trying to deploy both small and large scale Spark on Kube. Related Links DoKC Website - https://dok.community/ DoKC Meetups - https://www.meetup.com/data-on-kubernetes-community/ Join Slack - https://join.slack.com/t/dokcommunity/shared_invite/zt-1vgv7ymz7-YtLFvZicrcLP9fS3o_r2_w…

Data on Kubernetes Community

1
DoK @ Comcast - Deliver Business Outcomes & Improved DevX with Data Services on K8s | DoKC Town Hall 16:43

לפני 1 year16:43

16:43

DoK @ Comcast: Delivering Business Outcomes & Improved DevX with Data Services Running on Kubernetes Presented by Greg Otto, Executor Director, DevX Platforms & Charles Ju, Principal Engineer Transforming how to deliver measurable value using data on Kubernetes, while providing psychological safety. If you just sighed, you’re one of the many people leaders and engineers who know the challenges of making these changes. In this talk, we will share our transformation journey, the “Months to Minutes” outcomes we achieved, the architecture approach, and the human journey from one of our engineers. Related Links DoKC Website - https://dok.community/ DoKC Meetups - https://www.meetup.com/data-on-kubernetes-community/ Join Slack - https://join.slack.com/t/dokcommunity/shared_invite/zt-1vgv7ymz7-YtLFvZicrcLP9fS3o_r2_w…

Data on Kubernetes Community

1
DoK Talks - What is Kafka? The rise of one of the world's most used streaming data technologies // Abbey Russell 15:28

לפני 2 years15:28

15:28

Abbey Russell, PM at Cockroach Labs, shared the backstory on how and why Kafka was created. Along the way, you'll learn about - Who Franz Kafka was - Kafka's earliest use at Linkedin in 2010 - Why organizations like Uber/Coursera/Mailchimp use it today - Future of Data Streaming To find out more about how organizations are benefitting from running streaming workloads on Kubernetes, check out our 2022 research report- https://dok.community/data-on-kubernetes-2022-report/ https://dokcommunity.slack.com/ https://dok.community/ About Data on Kubernetes Community (DoKC) DoKC is an independent tech community where practitioners go to run data workloads on Kubernetes. It is managed by Constantia.io. We produce a range of resources for end users looking to run DoK including: - DoK Landscape An interactive landscape of products and services for running data workloads on Kubernetes. (https://dok.community/landscape/) - Videos and Blogs Hundreds of videos and blogs including how tos, tutorials, use cases, case studies, and more. (https://dok.community/landscape/) (https://dok.community/blog/) - DoK Reports DoKC works with an independent research firm to annually produce the industry’s only DoK-focused report. (https://dok.community/dok-reports/) - Livestreams and Local Meetups Weekly livestreams on DoK-related topics including demos, tutorials, visionary talks, panels, and more. (https://www.meetup.com/data-on-kubern...) - DoK Days @ KubeCon We host hybrid events twice a year for people to get together and learn in-person or online. Details are published on our main website. (https://dok.community/) Connect With Us: https://dok.community/ https://twitter.com/DoKcommunity https://www.linkedin.com/company/6667... https://github.com/dokc…

Data on Kubernetes Community

1
DoK Talks - (almost)Everything you need to know about stateful cloud native network applications // W Watson 43:39

לפני 2 years43:39

43:39

https://go.dok.community/slack https://dok.community/ https://youtu.be/KjiK6eXYO34 DoK Talk with W Watson, Founder at Vulk Co-op

Data on Kubernetes Community

1
The Outer Nerd #001 - Dungeons & Dragons - Why should you care? // Abhi Vaidyanatha, Fabian Met & Chase Christensen 58:25

לפני 3 years58:25

58:25

https://dokcommunity.slack.com/ https://dok.community/ ABSTRACT OF THE TALK Fabian, Chris and Abhi will discuss their passion for roleplaying games, and what they can teach us about the power of community, improvisation, and using our creativity.

Data on Kubernetes Community

1
DoK Talks #155 - Databases at the edge with K3s and ARM devices // Sergio Méndez 49:40

לפני 3 years49:40

49:40

https://go.dok.community/slack https://dok.community/ https://youtu.be/KjiK6eXYO34 ABSTRACT OF THE TALK In this talk Sergio is going to present different ways to store data at the edge using different databases and Long Horn as a storage class. All this running on a Raspberry Pi and showing and small application using a database running at the edge. BIO Sergio Méndez is a systems engineer and professor of operating systems at USAC Guatemala university. His work at the university is related to teaching and researching cloud native technologies with his students. He has experience working on DevOps, and MLOps using open source technologies at work. About open source communities, he is involved in the CNCF Community, promoting students into the CNCF Ecosystem and hosting a Cloud Native meetup in Guatemala. He has been a speaker at several conferences such as KubeCon, WTFisCloudNative, and Kubernetes Community Days. He is also a Linkerd Ambassador and author of Edge Computing systems with Kubernetes with Packt publisher. KEY TAKE-AWAYS FROM THE TALK How to storage data at the edge using databases…

Data on Kubernetes Community

1
DoK Talks #154 - StatefulSets in K8 // Srinivas Karnati 31:55

לפני 3 years31:55

31:55

https://go.dok.community/slack https://dok.community/ Link: https://youtu.be/n_thXwyJNSU ABSTRACT OF THE TALK Deploying Stateless applications is easy but this is not the case for Stateful applications. StatefulSets are the K8s API object that helps to manage stateful application. Learn about what Stateful sets are, how to create, How it differs from Deployments. BIO Passionate about Cloud Native, Kubernetes, Developer relations KEY TAKE-AWAYS FROM THE TALK This talk is focused on basics of StatefulSet, how StatefulSet differs from Deployments, How to manage Stateful app using StatefulSet…

Data on Kubernetes Community

1
Data-driven Diversity, Equity, and Inclusion // Lisa-Marie Namphy, Melissa Logan, Tiffany Jachja, Audra Montenegro & Cortney Nickerson (DoK Day North America 2022) 19:50

לפני 3 years19:50

19:50

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY )

Data on Kubernetes Community

1
Formula 1 telemetry processing using Apache Kafka on Kubernetes // Paolo Patierno (DoK Day North America 2022) 15:36

לפני 3 years15:36

15:36

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY ) Video - https://youtu.be/4cPVRWOK-_E ABSTRACT Apache Kafka is the de facto data streaming platform used for ingesting vast amounts of data and processing them in real-time. Low latency analytics are vital if users are to react to events as fast as possible and to effectively shape future decision making. Together with Kubernetes, it allows to develop cloud oriented analytics solutions which are highly scalable. During this session, we will explore an Apache Kafka based solution running on Kubernetes to help a Formula 1 team gain insights during a race. The “real” data will come from a well-known racing game and will be processed by our application in real-time, making us feel like real track-side F1 engineers!…

Data on Kubernetes Community

1
Choosing Kubernetes for Stateful Applications // Akshay Ram & Peter Schuurman (DoK Day North America 2022) 18:31

לפני 3 years18:31

18:31

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY ) Video - https://youtu.be/Y4tdy9lctEI ABSTRACT Learn how customers are increasingly deploying stateful applications on Kubernetes to benefit from portability, economies of scale, and built-in orchestration capabilities. This talk will include how customers choose between using Kuberentes, or a data Software as a Service (SaaS) and stateful capabilities of Kubernetes across two dimensions - the application orchestration and the storage layer. Also learn about MariaDB SKYSQL, a database software as a service that runs thousands of StatefulSet Pods across multiple zones and regions on Kubernetes.…

Data on Kubernetes Community

1
Kubernetes 360º - Data driven observability - from Secrets to logs // Ben Hirschberg (DoK Day North America 2022) 17:11

לפני 3 years17:11

17:11

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY ) Video - https://youtu.be/A1ch4AhKoeQ ABSTRACT If there’s one thing that everyone can agree on - it’s that the sheer scale and complexity of Kubernetes operations is growing constantly. What’s more, cloud native environments are becoming more and more expensive to operate and manage, as well as increasingly difficult to secure. On the bright side, there is a growing ecosystem of exceptional open source tools to help overcome this complexity, and provide greater situational awareness to what’s happening in your many and multiple Kubernetes clusters. In this talk, we’ll take a deep dive on how to understand everything that is happening with stateful workloads in your Kubernetes clusters from health to security - including the management and monitoring of your clusters, troubleshooting and debugging, through full security scanning in multiple data perspectives- from configMaps, Secrets, hostPaths and many more security driven checks for proper role-based access control, vulnerabilities and misconfigurations. We will demonstrate how you can have a 360º-view of your Kubernetes clusters in a single open source dashboard leveraging tools like Lens and Kubescape, and ultimately enforce policies with Open Policy Agent.…

Data on Kubernetes Community

1
Shifting Left Stateful Applications In Kubernetes // Viktor Farcic (DoK Day North America 2022) 15:52

לפני 3 years15:52

15:52

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY ) Video - https://youtu.be/LymPjH6HA3E ABSTRACT Stateless apps are easy to manage. More often than not, a Kubernetes Deployment, with a Service, Ingress, and Horizontal Pod Autoscaler (HPA) is enough. Almost everyone can do it. But, when it comes to stateful applications, things become a bit more complicated. We might need a database and storage. We might need to manage database users and schema. We might need to consider quite a few other things. Stateful apps are harder for everyone, especially if we want to shift left and enable developers to do it themselves. In this talk, we'll try to make the management of stateful applications easy for everyone. We'll accomplish that by creating easy-to-consume services that are made specifically for the needs of our organizations. We'll see how to create new Kubernetes Custom Resource Definitions (CRDs) and controllers using Crossplane. Those controllers will envelop all the tools, resources, and processes we might need. As a result, ops can focus on creating such services while everyone else can consume them (create and manage everything related to stateful apps) without opening JIRA tickets and waiting for others to complete their tasks.…

Data on Kubernetes Community

1
Medical - Healthcare Data on Kubernetes // Olyvia Rakshit & Prasad Dorbala (DoK Day North America 2022) 13:41

לפני 3 years13:41

13:41

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY ) ABSTRACT Healthcare organizations are transforming their applications and embracing digital platforms for efficient patient care. Today, compute at the edge, plays a critical role in deploying innovative healthcare applications that promise new approaches to patient care. Connected care platforms at scale are turning to Kubernetes infrastructure for deployment of telehealth and telemedicine apps. These kinds of applications often need real time image processing and inferencing at the edge, which in turn, require a distributed application platform on Kubernetes. They also need secure foundations for a regulatory compliant framework for wider adoption. To elaborate further, a distributed platform or a multi-cluster infrastructure is necessary for healthcare applications. All medical application platforms are data intense. For eg. Medical applications that need to stream video data, i.e move large quantities of data need secure connections from edge to cloud. Smart healthcare applications are also seeing the value of multi-cloud deployments. The ability to choose different technologies like AI, data storage systems or some specific relational database systems is driving multi-cloud adoption in healthcare solutions. A multi-cluster connectivity fabric is thus needed for data intensive healthcare applications to distribute across geographical boundaries. The fabric needs to have inbuilt automation and the smarts to make the best use of compute resources in a distributed infrastructure, for improved patient healthcare.…

Data on Kubernetes Community

1
Highly Available Postgres Clusters In Kubernetes // John Long & Jonathan Gonzalez (DoK Day North America 2022) 15:04

לפני 3 years15:04

15:04

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY ) ABSTRACT A practical session about running Highly Available PostgreSQL in Kubernetes. The primary objective will be to demonstrate how to set up a reliable architecture in a Kubernetes cluster to achieve low RTO and RPO. This will be covered by going over the various Kubernetes native components of our fully open source operator CloudNativePG[0]. It will be demonstrated how these components allow for a simple adoption of PostgreSQL into Kubernetes environments. If time and technical constraints permit there will be a live demo at the end of the presentation; if for some reason that is not possible there will be a link to a recorded demo. https://cloudnative-pg.io/…

Data on Kubernetes Community

1
Inter-Cluster PostreSQL on Kubernetes // Julian Fischer (DoK Day North America 2022) 17:07

לפני 3 years17:07

17:07

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY ) ABSTRACT In this talk you’ll explore how to run a PostgreSQL cluster across multiple Kubernetes clusters. Learn what challenges arise when using asynchronous streaming replication in a set of Kubernetes clusters spanning across several geographical regions. It will be discussed how such a topology can be established using Kubernetes and Kubernetes Add-Ons. You will learn which qualities the system has, where such a distributed system plays its strengths and where it may be problematic to use it. At the end of the talk you will have an increased understanding about how to create multi-regional data service instances using PostgreSQL as an example.…

Data on Kubernetes Community

1
Open Source Databases on Kubernetes- Best Practices // Peter Zaitsev (DoK Day North America 2022) 16:04

לפני 3 years16:04

16:04

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY ) ABSTRACT So you’re looking to run your Open Source Database on Kubernetes. What best practices should you follow and what pitfalls should you avoid ? In this presentation we will look at how to run stateful applications on Kubernetes overall as well as what is particularly important for databases - we will cover high availability, security, backups and disaster recovery. Finally we will show how these practices can be implemented with Percona Operators for MySQL, MongoDB, PostgreSQL - one of the leading solutions to run Open Source Databases on Kubernetes…

Data on Kubernetes Community

1
The Kubernetes Native Database // Jeffrey Carpenter (DoK Day North America 2022) 16:26

לפני 3 years16:26

16:26

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY ) ABSTRACT In the software industry we’re fond of terms that define major trends, like “cloud native”, “Kubernetes native” and “serverless”. As more and more organizations move stateful workloads to Kubernetes, we’ve started to see these terms applied to data infrastructure, where they can get overtaken by marketing hype unless we work to define them. In this talk, we’ll examine two different databases, TiDB and Apache Cassandra, in order to identify what it means for a database to be Kubernetes native and why it matters. We’ll look at points including: - The differences between cloud native, Kubernetes native, and serverless - How databases become Kubernetes native - Benefits of Kubernetes native databases - How Kubernetes can better support databases…

Data on Kubernetes Community

1
Databases on Kubernetes: Why are they important? // With Bhavin Shah, Xing Yang, Gabriele Bartolini & Patrick McFadin (DoK Day North America 2022) 34:51

לפני 3 years34:51

34:51

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY ) ABSTRACT Kubernetes has crossed the chasm, but what about stateful applications and databases? Join us for this panel discussion and learn more about how organizations are deploying different databases like PostgreSQL and Cassandra on Kubernetes, what are the benefits of running databases on Kubernetes, and how the ecosystem is working towards making these things boring, so you can focus on your applications! We will have an interactive discussion with the hosts and guests of the Kubernetes Bytes podcast, and open it up to the audience to ask questions and learn more about the what, why, and how about Databases on Kubernetes!…

Data on Kubernetes Community

1
Data streaming on Kubernetes // Yaniv Ben Hemo (DoK Day North America 2022) 13:51

לפני 3 years13:51

13:51

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY ) ABSTRACT I will cover what is the current data streaming on k8s landscape, why it is important, use cases, and what are the challenges needed to solve

Data on Kubernetes Community

1
Architecting Your First Event Driven Serverless Streaming Applications on K8 // Timothy Spann (DoK Day North America 2022) 13:29

לפני 3 years13:29

13:29

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY ) ABSTRACT Once you have built a topic in Apache Pulsar, you will quickly see the need to build event-driven applications. This can require a lot of decisions on what framework to use, where to run it, how to deploy it, and how to manage these applications on Kubernetes cloud natively. I will walk you through step-by-step in building Pulsar Functions which is the easy way to design, test, develop, integrate, deploy, monitor, and manage serverless streaming applications in Java and Python. Together we will build a full application as an Apache Pulsar function and enjoy the power of running it in the cloud for IoT events and add any routing, transformation, or machine learning that we need to accomplish our business requirements. Through FunctionMesh we run on Kubernetes natively. In this talk, you will deploy ML functions to transform real-time data on Kubernetes.…

Data on Kubernetes Community

1
Fybrik - A Kubernetes based platform for governed data use // Flora Gilboa-Solomon, Alexey Roytman, Maryna Strelchuk & Barry Hijkoop (DoK Day North America 2022) 20:59

לפני 3 years20:59

20:59

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY ) ABSTRACT Data is the foundation for business value. However, in many enterprises, it is spread across different data stores, public/private clouds, and on-premises. The use of data is governed by regulatory requirements and enterprise policies and enterprises face dynamic data residency challenges. Fybrik ( https://fybrik.io ) is an open-source, Kubernetes-based infrastructure that enables easier access to data distributed across hybrid and multi-cloud environments, while enforcing data governance policy according to the business’s needs. In this talk, we will share use cases from a pilot project carried out with ING (a multinational banking corporation). Fybrik enabled ING the use of their data across geographical borders, across clusters in a secure, automated, and compliant manner.…

Data on Kubernetes Community

1
The Challenges of Data Processing On Kubernetes - A look at Spark, Flink, Dask, and Ray // Holden Karau (DoK Day North America 2022) 20:09

לפני 3 years20:09

20:09

From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY) ABSTRACT This talk will go through both the improvements that have been made in Kubernetes for batch analytic workloads as well as some of the current pain experienced by users and developers moving their workloads to Kube. In this talk you will learn about how we “cheated” back in the YARN and Mesos days to make things go fast, why Kubernetes doesn’t like those cheats, and what some alternatives are.…

Data on Kubernetes Community

1
Scaling our SaaS offering to thousands of clusters // Dax McDonald (DoK Day North America 2022) 21:04

לפני 3 years21:04

21:04

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY ) ABSTRACT Sourcegraph is a code intelligence platform that helps our customers to understand their code better. As we have scaled up, we are starting to run hundreds of instances for our customers in separate kubernetes clusters. Running dozens of distinct clusters with a stateful application that scales horizontally and vertically has been challenging. Sourcegraph's inherently stateful nature requires thoughtful planning to achieve resilience while making tradeoffs to keep costs down. This talk will discuss our methodologies to allow our cluster to scale while minimizing drift across multiple customer deployments. I will also cover our use of Terraform, ArgoCD, and Helm to manage the application and infrastructure. Finally, this talk will discuss our disaster recovery strategies that allowed our organization to achieve SOCII compliance while iterating quickly. If you are looking to scale out a managed offering at your organization, learn from our mistakes and come to this talk!…

Data on Kubernetes Community

1
Why we decided to migrate our Jaeger storage to ClickHouse on Kubernetes // Arul Jegadish Francis (DoK Day North America 2022) 13:48

לפני 3 years13:48

13:48

From the DoK Day North America 2022 ( https://youtu.be/YWTa-DiVljY ) Abstract We at OpsVerse provide a DevOps tools platform with fully-managed open source-based tools. One of our key offerings is a holistic observability platform. Metrics and logs are straightforward to aggregate, however traces – which are collected using CNCF Jaeger – were left with some holes in advanced insights. In this talk, I will explain why we decided to switch our Jaeger storage backend to ClickHouse, and why we decided to run it on Kubernetes. I will also share tips and tricks to migrate and run ClickHouse on Kubernetes.…

Data on Kubernetes Community

1
Building a Digital Factory for the Sheet Metal Industry // Elie Assi (From the DoK Day North America 2022) 20:48

לפני 3 years20:48

20:48

From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY) Abstract We develop systems to digitize the sheet metal industry with the belief that they should cooperate with each other in an open way. We are convinced that the future lies in creating a software ecosystem that interconnects all levels of the company and even manages to communicate with supplier and customer systems, making for more agile management throughout the entire value chain. One of our newer projects is providing a multi-tenant cloud service to our clients. The chosen database for the service we are developing is MongoDB. After testing with docker containers on virtual machines, we decided to finally host the service in Kubernetes because we wanted something to be able to scale at large for our clients being a service that will be consumed by a multitude of our clients, so to the database in order to reduce latency.…

Data on Kubernetes Community

1
How we built our Big Data Stack (almost) entirely on top of Kubernetes // Neylson Crepalde (From DoK Day NA 2022) 16:00

לפני 3 years16:00

16:00

From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY) Abstract Working with Terabytes of data is a major challenge for organizations both in terms of architecture and cost. In recent years, a new paradigm has emerged in the world of Big Data, that is, implementing the entire architecture for processing massive data from a microservices approach. This architecture proved to be scalable, efficient, with excellent performance, failsafe, portable between different cloud providers and at a lower cost than conventional architectures. In this talk, I will present how my team managed to implement this architecture based on Kubernetes and what were the main gains we had with the innovation.…

Data on Kubernetes Community

1
Dok Talks #153 - CRD Panel // Eyar Zilberman & Álvaro Hernández 58:05

לפני 3 years58:05

58:05

https://go.dok.community/slack https://dok.community We are going to speak about CRDs, and discuss considering them as higher level entities that we normally consider them. CRDs normally are kind of a byproduct of an operator. But in reality, they can be considered as the user-facing API of the operator surface. And as such, we would like to introduce conversations about even standardizing some of them, or creating "specifications" out of some CRDs. Take this for example: there are some CRDs that could be, potentially, common across multiple operators. Think of a CRD to represent an S3 (or any other storage object) bucket. For operators that store data in object stores, each and every one of them are creating their own versions. But shouldn't we have some "standardized" `ObjectStorage` CRD, without a controller, that all could use? We'll be joined by John Sanda of DataStax who works on the K8ssandra project, Alvaro Hernandez who is the CEO of Ongres and Postgres superhero, and Eyar Zilberman Co-Founder and Chief Product Officer at Datree.…

Data on Kubernetes Community

1
Dok #152-Running PostgreSQL in Kubernetes:from day 0 to day 2 with CloudNativePG // Gabriele Bartolini 1:03:50

לפני 3 years1:03:50

1:03:50

https://go.dok.community/slack https://dok.community With: Gabriele Bartolini - Vice President/CTO of Cloud Native and Kubernetes, EDB Bart Farrell - Head of Community, Data on Kubernetes Community ABSTRACT OF THE TALK Imagine this: you have a virtual infrastructure based on Kubernetes, made up of virtual data centers, possibly spread across multiple Kubernetes clusters and regions. Your infrastructure could even be hosted on premises or on different cloud service providers. Infrastructure as Code is a requirement. You’ve been tasked to run Postgres databases, alongside your applications. The good news is that you can leverage a fully open source stack with Kubernetes, PostgreSQL and the CloudNativePG operator, and deploy your Postgres database in the same way you deploy applications. Join me in this webinar to discover the key role that you have to make this succeed, starting from day 0 through day 2 operations. I’ll share some examples and best practices for running Postgres databases in Kubernetes, before peeking at the new features we are developing for the months to come. BIO A long time open source programmer and entrepreneur, Gabriele has a degree in Statistics from the University of Florence. After having consistently contributed to the growth of 2ndQuadrant and its members through nurturing a lean and devops culture, he is now leading the Cloud Native initiative at EDB. Gabriele lives in Prato, a small but vibrant city located in the northern part of Tuscany, Italy - known for having hosted the first European PostgreSQL conferences. His second home is Melbourne, Australia, where he studied at Monash University and worked in the ICT sector. He loves playing the Blues with his Fender Stratocaster, but his major passions are called Elisabeth and Charlotte!…

Data on Kubernetes Community

1
Dok Talks #148 - Cost and Kubernetes // Chris Love 45:25

לפני 3 years45:25

45:25

https://go.dok.community/slack https://dok.community With: Chris Love - Managing Partner, LionKube Bart Farrell - Head of Community, Data on Kubernetes Community ABSTRACT OF THE TALK Using Kubernetes to run data workloads costs less than running the same workloads on separate servers. But how do we save at least twenty to thirty percent more? We need the data and a cost-savings culture. We will explore a project called Open Cost: cost monitoring for cloud-native environments. Secondly, we will explore why company culture is more important than cost savings tools. BIO Chris Love is a Google Cloud Certified hybrid Multi-cloud Fellow and a Co-founder of Lionkube. Chris co-authored the book Core Kubernetes. He has over 25 years of software and IT engineering experience with companies including Google, Oracle, VMWare, Cisco, Johnson & Johnson, and others. As a thought leader within Kubernetes and the DevOps community, Chris Love has contributed to many open source projects, including Kubernetes, kops (former AWS SIG lead), Bazel (contributed to Kubernetes rules), and Terraform (an early contributor to the VMware plugin). His professional interests include Kubernetes, IT culture transformation, containerization technologies, automated testing frameworks and practices, and DevOps. Love also enjoys speaking around the world about DevOps, Kubernetes, and Technology and mentoring people in the IT and Software industry. Outside of work Love enjoys skiing, volleyball, yoga, and other outdoor activities that come with living in Colorado. He’s also been a practicing martial artist for over 20 years. If you’re interested in having virtual coffee or have questions for Chris, you can contact him at @chrislovecnm on Twitter or LinkedIn. KEY TAKE-AWAYS Two things will save us money running a stateful workload hosted in Kubernetes; data and company culture.…

Data on Kubernetes Community

1
Dok Talks #151 - Analytics with Apache Superset and ClickHouse // Vijay Anand Ramakrishnan 33:00

לפני 3 years33:00

33:00

https://go.dok.community/slack https://dok.community With: Vijay Anand Ramakrishnan - Database Administrator, ChistaDATA Bart Farrell - Head of Community, Data on Kubernetes Community ABSTRACT OF THE TALK This talk concerns performing analytical tasks with Apache Superset with ClickHouse as the data backend. ClickHouse is a super fast database for analytical tasks, and Apache Superset is an Apache Software foundation project meant for data visualization and exploration. Performing analytical tasks using this combo is super fast since both the software are designed to be scalable and capable of handling data of petabyte scale. BIO Vijay Anand is based out of Chennai (India), working as a Database Administrator in ChistaDATA. He has extensive experience in ClickHouse, Python and has contributed as a technical lead in multiple organizations building ClickHouse based solutions. His areas of interest include database design, building software solutions using open source technologies. He is the author of a book on ClickHouse titled "Up and Running with ClickHouse". KEY TAKE-AWAYS Real time analytics, Data exploration and Visualization…

Data on Kubernetes Community

1
Dok Talks #150 - Building a Simple Postgres Async Streaming Cluster // Julian Fischer 1:04:45

לפני 3 years1:04:45

1:04:45

https://go.dok.community/slack https://dok.community With: Julian Fischer - CEO, anynines GmbH Bart Farrell - Head of Community, Data on Kubernetes Community ABSTRACT OF THE TALK In this talk you will learn how to build the a Postgres service with Kubernetes. See how asynchronous replication is set up using a Kubernetes resources including, a headless Service, StatefulSet, ConfigMap and Secret. BIO Julian Fischer, CEO of anynines, has dedicated his career to the improvement and automation of software operations. In more than fifteen years,he has built several application platforms with various automation tools. His latest passions are data service automation, Cloud Foundry, BOSH and Kubernetes. KEY TAKE-AWAYS At the end of the talk you will know the Kubernetes resources a controller had to manage as input to building an actual Operator.…

Data on Kubernetes Community

1
DoK Talks #149 - Overcoming challenges with protecting and migrating data in multi-cloud K8s environments // Sebastian Glab & Martin Phan 47:40

לפני 3 years47:40

47:40

https://go.dok.community/slack https://dok.community/ With: Sebastian Glab - Cloud Architect, CloudCasa by Catalogic Martin Phan - Field CTO – Americas, CloudCasa by Catalogic Bart Farrell - Head of Community, Data on Kubernetes Community ABSTRACT OF THE TALK If you are running or planning a multi-cloud or even a multi-cluster environment, there are several considerations in implementing a data protection solution – especially if you plan on an organic home-grown, do-it-yourself option. This talk will highlight challenges and best practices around centralized management of configuration, credentials, compliance across multiple accounts, regions, providers etc. We will also highlight the deviations in CSI driver implementations of various storage vendors and cloud providers. Finally, we will cover the various recovery options available in the market today. Kubernetes cloud services are popular since they mitigate, but do not eliminate, the difficulties of operating a Kubernetes environment. This is especially true for protecting the stateful configuration and data of your Kubernetes applications, where the inherent high-availability and infrastructure as code are not a substitute for have cloud-native backup and disaster recovery capabilities. Further, many companies now have multi-cloud strategies for their cloud-native applications. These challenges can be addressed with backup applications that are both Kubernetes managed service and multi-cloud aware in order to snapshot, copy, restore, and migrate Kubernetes workloads (resources and data) running on AKS, EKS and GKE. Capturing information from cloud accounts and how the cluster and storage resources are configured allows 1) centralized visibility into all cloud accounts and the clusters and resources in the accounts including for compliance; 2) cross-account, cross-cluster, and cross-region data restores; 3) automation of the cluster and data restores including for Dev, Test, and Production recovery use cases. BIO Sebastian Glab is a Cloud Architect for CloudCasa and he resides in Poland. He is responsible for integrating the different cloud providers with the CloudCasa service, and making sure that all clusters in the cloud service get discovered and protected. In his free time, he plays volleyball and develops his own projects. Martin Phan is the Field CTO in North America for CloudCasa by Catalogic Software. With over 20+ years of experience in the software-industry, he takes pride in supporting, developing, implementing, and selling enterprise software and data protection solutions to help customer solve their backup and recovery challenges. KEY TAKE-AWAYS FROM THE TALK 1) Challenges and best practices around centralized management of configuration, credentials, compliance across multiple accounts, regions, providers etc. 2) Advantages of cloud awareness and Kubernetes managed service awareness for application and data recovery and security 3) Examples of overcoming Container Storage Interface (CSI) deviations 4) Various recovery options available in the market today.…

Data on Kubernetes Community

1
DoK Talks #147 - Evaluating Cloud Native Storage Vendors // Dinesh Majrekar 1:00:03

לפני 3 years1:00:03

1:00:03

https://go.dok.community/slack https://dok.community/ With: Dinesh Majrekar - CTO, Civo Bart Farrell - Head of Community, Data on Kubernetes Community ABSTRACT OF THE TALK In a continuation of a talk given at DoK day at KubeCon EU 2022, join Dinesh Majrekar, Civo's CTO as they walk through their evaluation process of the CNCF Storage market. Civo offers managed Kubernetes clusters powered by K3s to customers around the world. We manage thousands of Virtual Machines and stateful customer data within multiple data centres across several continents. In late 2021, Civo had the opportunity to evaluate the CNCF storage landscape to move to a new technology stack. During the migration project, Civo evaluated Mayastor, Ondat, Ceph and Longhorn against the following metrics: Scalability Performance Ease of Support Attendants will see practical examples on how they could carry out their own similar evaluation and see some of the results of the Civo research project. BIO Dinesh is CTO at Civo. Having worked in the hosting industry for many years, Dinesh has a passion for creating solutions that operate at scale. This not only applies to the technology stack, but for nurturing engineers through their career.…

Data on Kubernetes Community

1
Dok Talks #146 - OpenFeature - Making feature flags a commodity // Oleg Nenashev 1:01:30

לפני 3 years1:01:30

1:01:30

https://go.dok.community/slack https://dok.community/ With: Oleg Nenashev - Community Builder and Developer Advocate, Dynatrace Bart Farrell - Head of Community, Data on Kubernetes Community ABSTRACT OF THE TALK With the evolution of software delivery methods, feature flags became one of the ways to deliver features and get user feedback. Modern applications may have hundreds of feature flags describing customizations, preview features and environment specifics. Feature flags are not just a single toggle anymore, their values are often determined at runtime depending on the instance, user context and inputs from the control plane. In this session we will talk about OpenFeature - a new collaboration by many feature flag vendors and end users. OpenFeature, a CNCF sandbox project, is an open standard for feature flag management, created to support a robust feature flag ecosystem using cloud native technologies and to basically make feature flags a commodity. OpenFeature provides a unified API and SDKs, and a developer-first, cloud-native implementation, with extensibility for open source and commercial offerings. BIO Oleg works on open source programs and Keptn community at the Dynatrace Open Source Program Office. He is a TOC chair and community ambassador in the Continuous Delivery Foundation. Oleg is a core maintainer and board member in Jenkins where he writes code, mentors contributors, and organizes community events and Google Summer of Code in the project. Open source software and open hardware advocate. Oleg has a PhD degree in hardware design and volunteers in the Free and Open Source Silicon Foundation. KEY TAKE-AWAYS FROM THE TALK Feature flags at scale 101. Feature flags are an important topic for stateful apps in K8s, and for progressive delivery and other modern approaches. OpenFeature is like OTel but for feature flags. You can try it now, here is a live demo to follow.…

Data on Kubernetes Community

1
DoK Talks #145 - Making Hard Things Easy is Hard // Kurt Rinehart 57:40

לפני 3 years57:40

57:40

https://go.dok.community/slack https://dok.community/ https://youtu.be/6eSWOUzCb4w With: Kurt Rinehart - Director of Information Engineering, Section Bart Farrell - Head of Community, Data on Kubernetes Community ABSTRACT OF THE TALK Making complicated things simple is a prime objective of technology. In addition to technical challenges there is an interfacing challenge. At Section we're automating all dimensions of hosting containerized workloads and solving the "location challenge" is one aspect of this. Making it simple to get the locations you want evokes many trade-offs and can collide with past habits that may no longer be suitable. BIO Dr. Kurt Rinehart is the Director of Information Engineering at Section. Kurt holds a PhD in Natural Resource Management, specialising in applying quantitative decision analysis to complex conservation problems. After working as a quantitative consultant, Kurt found and fell in love with startup life and Section. His current responsibilities include designing, developing, and managing autonomic computing systems and providing analytics and data science support across every facet of Section. Kurt is married with 2 kids and a dog, and plays guitar for The Intolerables (theintolerables.com). KEY TAKE-AWAYS FROM THE TALK - Good agile practice and feedback loops is critical. - "Trust but verify" is a good philosophy to keep in mind. - When developers are your users, there are extra challenges in finding the happy balance.…

Data on Kubernetes Community

1
DoK Talks #144 - We will Dok You! - The journey to adopt stateful workloads on k8s // Guy Menahem 1:06:30

לפני 3 years1:06:30

1:06:30

https://go.dok.community/slack https://dok.community/ https://youtu.be/AjvwG53yLMY With: Guy Menahem - Solution Architect, Komodor Bart Farrell - Head of Community, Data on Kubernetes Community ABSTRACT OF THE TALK Stateful workloads are the heart of any application, yet they remain confusing and complicated even to daily K8s practitioners. That’s why many organizations shy away from migrating their data - their prized possession - to the unfamiliar stateful realm of Kubernetes. After meeting with many organizations in the adoption phase, I discovered what works best, what to avoid, and how critical it is to gain confidence and the right knowledge in order to successfully adopt stateful workloads. In this talk I will demonstrate how to optimally adopt Kubernetes and stateful workloads in a few steps, based on what I’ve learned from observing dozens of different adoption journeys. If you are taking your first steps in data on K8s or contemplating where to start - this talk is for you! BIO - A Developer turned Solution Architect. - Working at Komodor, a startup building the first K8s-native troubleshooting platform. - Love everything in infrastructure: storage, networks & security - from 70’s era mainframes to cloud-native. - All about “plan well, sleep well”. KEY TAKE-AWAYS FROM THE TALK - Understand how critical stateful workloads are for any system, and that the key challenges to migrating it to Kubernetes are knowledge and confidence. - How to build the foundational knowledge required to overcome adoption challenges by creating a learning path for individuals and teams. - How to gain confidence to run stateful workloads on Kubernetes with support from the community (and yourself!)…

Data on Kubernetes Community

1
DoK Talks #142 - Kubernetes Cluster Upgrade Strategies and Data: Best Practices for your Stateful Workload // Peter Schuurman 58:45

לפני 3 years58:45

58:45

https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK How do you make sure your Stateful Workloads remain available when your Kubernetes infrastructure updates? This talk will discuss different strategies of upgrading a Kubernetes cluster, and how you can manage risk for your workload. The talk will showcase demos of each upgrade strategy. BIO Peter is a Senior Software Engineer on GKE at Google. He works on improving Kubernetes for Stateful workloads. His main focus is on enhancing the Kubernetes ecosystem for high availability applications. KEY TAKE-AWAYS FROM THE TALK The mechanics of different upgrade strategies, when to apply a particular upgrade strategy depending on your Stateful workload and how to mitigate risk to your application’s availability.…

Data on Kubernetes Community

1
DoK Talks #144 - Mastering MongoDB on Kubernetes, the power of operators // Arek Borucki 1:00:50

לפני 3 years1:00:50

1:00:50

https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK During my first talk for DoK community I want to walk you through the world of NoSQL database MongoDB and Kubernetes Operators - Community Edition, Enterprise Edition (MongoDB and Ops Manager on K8s), and Atlas operator, highlight the most important capabilities, talk about use cases and challenges, the theory will be mixed with a live demos! BIO I'm a SRE / NoSQL / DevOps professional. I hold CKA, CKAD, CKS, also I’m MongoDB Certified DBA and MongoDB Champion. I have experience with multiple cloud providers, Kubernetes, different types of K8s operators (Strimzi, RabbitMQ Cluster Operator), but especially MongoDB K8s Operator. I also work with KEDA. Since 2017, I have been a speaker at MongoDB conferences all around the world (USA, China, Europe). KEY TAKE-AWAYS FROM THE TALK I would like to share the best practices of running NoSQL database - MongoDB on Kubernetes also I want to show how to manage Atlas (MongoDB cloud) via K8s operator https://www.mongodb.com/developer/community-champions/arkadiusz-borucki/…

Data on Kubernetes Community

1
DoK Specials - Why are Operators paramount to running stateful workloads on Kubernetes? 53:36

לפני 3 years53:36

53:36

In this panel with Sylvain Kalache, Head of Content at the DoK Community, drives a conversation featuring Nic Vermandé- Principal Developer Advocate at Ondat, Julian Fischer- CEO at anynines, and Sergey Pronin- Group Product Manager at Percona.

Data on Kubernetes Community

1
DoK Talks #141 - Dossier: multi-tenant distributed Jupyter Notebooks // Iacoppo Colonnelli & Dario Tranchitella 1:00:10

לפני 3 years1:00:10

1:00:10

https://go.dok.community/slack https://dok.community ABSTRACT OF THE TALK When providing data analysis as a service, one must tackle several problems. Data privacy and protection by design are crucial when working on sensitive data. Performance and scalability are fundamental for compute-intensive workloads, e.g. training Deep Neural Networks. User-friendly interfaces and fast prototyping tools are essential to allow domain experts to experiment with new techniques. Portability and reproducibility are necessary to assess the actual value of results. Kubernetes is the best platform to provide reliable, elastic, and maintainable services. However, Kubernetes alone is not enough to achieve large-scale multi-tenant reproducible data analysis. OOTB support for multi-tenancy is too rough, with only two levels of segregation (i.e. the single namespace or the entire cluster). Offloading computation to off-cluster resources is non-trivial and requires the user's manual configuration. Also, Jupyter Notebooks per se cannot provide much scalability (they execute locally and sequentially) and reproducibility (users can run cells in any order and any number of times). The Dossier platform allows system administrators to manage multi-tenant distributed Jupyter Notebooks at the cluster level in the Kubernetes way, i.e. through CRDs. Namespaces are aggregated in Tenants, and all security and accountability aspects are managed at that level. Each Notebook spawns into a user-dedicated namespace, subject to all Tenant-level constraints. Users can rely on provisioned resources, either in-cluster worker nodes or external resources like HPC facilities. Plus, they can plug their computing nodes in a BYOD fashion. Notebooks are interpreted as distributed workflows, where each cell is a task that one can offload to a different location in charge of its execution. BIO Iacopo Colonnelli is a Computer Science research fellow. He received his Ph.D. with honours in Modeling and Data Science at Università di Torino with a thesis on novel workflow models for heterogeneous distributed systems, and his master’s degree in Computer Engineering from Politecnico di Torino with a thesis on a high-performance parallel tracking algorithm for the ALICE experiment at CERN. His research focuses on both statistical and computational aspects of data analysis at large scale and on workflow modeling and management in heterogeneous distributed architectures. Dario is an SWE that turned DevOps, and he's regretting this choice day by day. Besides making memes on Twitter that gain more reactions than technical discussions, leading the development of Open Source projects at CLASTIX, an Open Source-based start-up focusing on Multi-Tenancy in Kubernetes. KEY TAKE-AWAYS FROM THE TALK From this talk, people will learn: - The different requirements of Data analysis as a service - How to configure for multi-tenancy at the cluster level with Capsule - How to write distributed workflows as Notebooks with Jupyter Workflows - How to combine all these aspects into a single platform: Dossier All the software presented in the talk is OpenSource, so attendees can directly play with them and include them in their experiments with no additional restrictions.…

Data on Kubernetes Community

1
DoK Talks #140 - Data protection of stateful environment // Timothy Dewin 42:35

לפני 3 years42:35

42:35

https://go.dok.community/slack https://dok.community ABSTRACT OF THE TALK More and more we see stateful workloads pop up in Kubernetes clusters. These workloads generate data that is unique and is ephemeral. During this talk we will discuss the challenges of stateful workloads and how you can successfully protect BIO Working over a decade in IT as a technical expert for Veeam Software. Specializing in backup for the modern hybrid cloud. Passion for scripting and programming. Husband of Lena, father of Lev. KEY TAKE-AWAYS FROM THE TALK Data backup of Kubernetes. DR for Kubernetes…

Data on Kubernetes Community

1
DoK Talks #139 - Private DBaaS on Kubernetes // Sergey Pronin 53:25

לפני 3 years53:25

53:25

https://go.dok.community/slack https://dok.community ABSTRACT OF THE TALK Percona is committed to deliver solutions to run open source databases anywhere without lock in. As part of this commitment, we have created Operators to run MySQL, PostgreSQL and MongoDB on Kubernetes. Learn how Percona Monitoring and Management (PMM) allows you to enable developers to deploy and manage databases anywhere with private Database-as-a-service capability backed by Operators. BIO Product and technology leader. Worked in various fields: internet service providers, financial sector and merge & acquisition business. Currently leads product @ Percona focusing on cloud native technologies for open source databases KEY TAKE-AWAYS FROM THE TALK Learn how Percona Monitoring and Management (PMM) allows you to enable developers to deploy and manage databases anywhere with private Database-as-a-service capability backed by Operators. You will get high level overview of Percona's Operators structure and how private DBaaS can boost the productivity of your engineering and IT teams.…

Data on Kubernetes Community

1
DoK Talks #138 - Build your own social media analytics with Apache Kafka // Jakub Scholz 56:25

לפני 3 years56:25

56:25

https://go.dok.community/slack https://dok.community ABSTRACT OF THE TALK Apache Kafka is more than just a messaging broker. It has a rich ecosystem of different components. There are connectors for importing and exporting data, different stream processing libraries, schema registries and a lot more. The first part of this talk will explain the Apache Kafka ecosystem and how the different components can be used to load data from social networks and use stream processing and machine learning to analyze them. The second part will show a demo running on Kubernetes which will use Kafka Connect to load data from Twitter and analyze them using the Kafka Streams API. After this talk, the attendees should be able to better understand the full advantages of the Apache Kafka ecosystem especially with focus on Kafka Connect and Kafka Streams API. And they should be also able to use these components on top of Kubernetes. BIO Jakub works at Red Hat as Senior Principal Software Engineer. He has long-term experience with messaging and currently focuses mainly on Apache Kafka and its integration with Kubernetes. He is one of the maintainers of the Strimzi project which provides tooling for running Apache Kafka on Kubernetes. Before joining Red Hat he worked as messaging and solution architect in the financial industry. KEY TAKE-AWAYS FROM THE TALK The key takeaway of this talk is that Apache Kafka is more than just a messaging broker. It is a platform and ecosystem of different components which can be used to solve complex tasks when dealing with events or processing data. The talk demonstrates this on loading tweets from Twitter and processing them using the different parts of the Kafka ecosystem. The whole talk and its demos are running on Kubernetes using the Strimzi project. So it also shows how to easily run all the different components on top of Kubernetes with the help of few simple YAML files.…

Data on Kubernetes Community

1
DoK Talks #137 - How to build your own “Doordash” app // Yaniv Ben Hemo 57:50

לפני 3 years57:50

57:50

https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK The entire app is built in microservices, running on k8s pods and uses k8s-native message broker called memphis WORKSHOP Learn how to build a cloud-native, distributed, event-driven food delivery application using a message broker, and deploy it to a kubernetes cluster. We will also discuss the importance of using a message broker within our app, and what are the benefits it provides to a more resilient application. The workshop is based on this article - https://medium.com/memphis-dev/how-to-build-your-own-wolt-app-b220d738bb71 Strongly advise to install before - memphis{dev} broker - https://github.com/memphisdev/memphis-brokerminikube - https://minikube.sigs.k8s.io/docs/start/node.js - https://nodejs.org/ If any questions arise, please raise a question on our Discord channel - https://discord.gg/WZpysvAeTf BIO Develop since I was 10 years old KEY TAKE-AWAYS FROM THE TALK Learn the basics of building and deploying cloud-native, distributed, event-driven app…

Data on Kubernetes Community

1
DoK Talks #136 - Building a mesh for databases from scratch and why // Maxwell Miao 47:45

לפני 3 years47:45

47:45

https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK In this talk, Maxwell is going to share his thoughts about Service Mesh and database operations, called Database Mesh, and figure out the design of Pisanix. Running databases on Kubernetes is hard, Database Mesh is building a new standard of databases management on cloud. It tries to make database operation more easier, and provides a framework for different kind of databases. Such as using a CRD called VirtualDatabase to describe a virtual database, developers will only need to know about this, meanwhile DBAs will have to bind TrafficStrategy and DatabaseEndpoint to VirtualDatabase, let Pisanix running like a MySQL proxy for applications. More features like SecurityClaim and TrafficQoS will be added to simplify the database operation work. BIO * Focus on SaaS and Database Mesh * Previously worked at 株式会社ネットスターズ、ChuanyangTech、AntFinancial、Yeepay * Linuxer & Gopher & Rustacean，open source advocator，cloud native envanglist * “CaaSOne” Community Cofounder * Contributor of multiple cloud native open source projects * Participated in writing Analysis and practice of Tensorflow、Istio Handbook KEY TAKE-AWAYS FROM THE TALK What's the difference of Service Mesh and Database Mesh. Why do people need a mesh for databases. What does Pisanix can help DBAs daily work. The current status of Pisanix.…

Data on Kubernetes Community

1
DoK Specials - Learn by doing in the DoK Community // Bart Farrell 15:55

לפני 3 years15:55

15:55

https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK As a community we are committed to making learning how to run stateful workloads on Kubernetes as accessible and inclusive as possible. Using our DoK Landscape which we launched at Kubecon, we will be doing a guided "tour" of the technologies featured on there. We will go one by one, providing tutorials with concrete instructions for you tp get hands on experience. At the end of each sprint we will share our learnings in public. BIO CNCF Ambassador. Data on Kubernetes (@Dokcommunity) Community Leader, amateur rapper, tech from the human side.…

Data on Kubernetes Community

1
DoK Talks #135 - DoK isn't just Database on Kubernetes // Patrick McFadin 46:00

לפני 3 years46:00

46:00

https://go.dok.community/slack https://dok.community ABSTRACT OF THE TALK What about your streaming and analytic workloads? If you are all-in on Kubernetes you can't forget about these important parts of your infrastructure. I'll talk about the current state of the art. Why organizations may hesitate to go beyond deploying databases in Kubernetes and most important, some key things you need to be successful. BIO Patrick McFadin is the co-author of the upcoming O’Reilly book “Managing Cloud-Native Data on Kubernetes” He currently works at DataStax in Developer Relations and as a contributor to the Apache Cassandra project. Patrick has worked as Chief Evangelist for Apache Cassandra and as a consultant for DataStax, where he had a great time building some of the largest deployments in production. Previous to DataStax, he held positions as Chief Architect, Engineering Lead and Database DBA/Developer. KEY TAKE-AWAYS FROM THE TALK People should walk away with a better understanding of what it takes to deploy streaming and analytic workloads in Kubernetes.…

Data on Kubernetes Community

1
DoK Talks #134 - Introducing CloudNativePG // Gabriele Bartolini & Leonardo Cecchi 1:05:20

לפני 3 years1:05:20

1:05:20

https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK CloudNativePG is an open source operator for the orchestration of Postgres workloads with a primary and an arbitrary number of read-only replicas that recently applied for the CNCF Sandbox. In this presentation we'll go through the main capabilities and run a live demo showing some of the main characteristics. BIO A long time open source programmer and entrepreneur, Gabriele has a degree in Statistics from the University of Florence. After having consistently contributed to the growth of 2ndQuadrant and its members through nurturing a lean and devops culture, he is now leading the Cloud Native initiative at EDB. Gabriele lives in Prato, a small but vibrant city located in the northern part of Tuscany, Italy - known for having hosted the first European PostgreSQL conferences. His second home is Melbourne, Australia, where he studied at Monash University and worked in the ICT sector. He loves playing the Blues with his Fender Stratocaster, but his major passions are called Elisabeth and Charlotte! Leonardo has been in computer science from the beginning, and programming has always been his obsession. The passion began when he was 6, with his father teaching him the rudiments of C programming. He is a long time PostgreSQL enthusiast, using it since 1998. He is one of the main contributors of the CloudNativePG project and a maintainer. In his free time, when he’s not having fun with Kubernetes and new programming languages, he’s a classical guitarist. KEY TAKE-AWAYS FROM THE TALK Run Postgres inside Kubernetes, alongside applications. Understand how to develop Cloud Native applications that rely on Postgres for data access through SQL. Understand how business continuity of Postgres databases can be done with CloudNativePG…

Data on Kubernetes Community

1
Dok Talks #133 - My First 90 days with Clickhouse // Alkin Tezuysal 47:20

לפני 3 years47:20

47:20

https://go.dok.community/slack https://dok.community ABSTRACT OF THE TALK This talk will tell the story of an analytics use case database from a non-OLAP and ACID-compliant RDBMS (MySQL) perspective. I will cover the basics of the Clickhouse database Sample Clickhouse installation in a lab environment. We are configuring Clickhouse for essential operations. We will load the sample data set and monitor it. We will query and visualize the results. This talk will also base on how Kubernetes can help Clickhouse implementation via an operator. Conclusions will include Do's and Don't of this emerging technology. Best practices and some advice around ingesting and analyzing terabytes of data efficiently. BIO Alkin Tezuysal has extensive experience in open source relational databases, working in various sectors for large corporations. With over 25 years of industry experience, he has acquired skills for managing large projects from the ground up to production. For the past decade, he's been focused on e-commerce, SaaS, and MySQL technologies. Alkin has managed and architected database topologies for high-volume sites. He has several years of experience in 24X7 support and operational tasks and improving database systems for major companies. He has led global operations teams on Tier 1/2/3 support for MySQL customers. He currently holds the position of EVP - Global Services at fast-growing startup ChistaDATA Inc. He's also co-author of the upcoming MySQL Cookbook 4th Edition. KEY TAKE-AWAYS FROM THE TALK Introduction to OLAP database from OLTP DBA…

Data on Kubernetes Community

1
DoK Specials - DEI Panel - We can do better 57:55

לפני 3 years57:55

57:55

https://go.dok.community/slack https://dok.community/ With: Melissa Logan - Director, Data on Kubernetes Lisa-Marie Namphy - Head of Developer Relations, Cockroach Labs Alexandra Rowell - Community Manager, SignalWire Cortney Nickerson - Developer Advocate, Kubeshop Iraide Ruiz - Engineering Manager, Twilio Tiffany Jachja - Engineering Manager, Vox Media Bart Farrell - Head of Community, Data on Kubernetes Community Diversity Equity and Inclusion can't just be words on a website or in an onboarding presentation. How can we make sure these are living, breathing pillars in our organizations? How can we do better? To take a closer look we got Melissa Logan, Director of the Data on Kubernetes, with an all-star panel to see what's happening and what can be improved.…

Data on Kubernetes Community

1
DoK Talks #132 - Time-series on SQL Server on Kubernetes on ARM64… without SQL Server! // Álvaro Hernández 1:05:15

לפני 3 years1:05:15

1:05:15

https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK Oh wow. What a weird title. Full of terms that don’t fit together. Or do they? This talk is for believers, those who believe that the author will deliver what the title says. Here’s the promise on what you will see: * Time-series specialized storage and functions on SQL Server. * SQL Server running on Kubernetes. * Using ARM64 (aarch64) nodes. * In reality, without using SQL Server, but rather Postgres Babelfish! The talk will be focused on a demo to show all these components working together. Surely the demo will fail badly, so get ready your rotten tomatoes! The good news is that you will be able to reproduce the demo on your environment too, as all the software used is open source and source code will be provided. BIO Álvaro is a passionate database and software developer. Founder of OnGres (one of the founding sponsors of DoK), he has been dedicated to PostgreSQL and R&D in databases for more than two decades. Álvaro is at heart an open source advocate and developer. He has created software like StackGres, a Platform for running Postgres on Kubernetes. As a well-known member of the PostgreSQL Community, Álvaro founded the non-profit Fundación PostgreSQL and the Spanish PostgreSQL User Group. He has contributed, among others, the SCRAM authentication library to the Postgres JDBC driver. You can find him frequently speaking at PostgreSQL, database, cloud (becoming an AWS Data Hero in 2019), and Java conferences. In the last 10 years, Álvaro has completed more than 100 tech talks ( https://aht.es ). KEY TAKE-AWAYS FROM THE TALK A key to progress is to combine technologies in novel ways to bring new capabilities to environments that typically lack them. For example, using Postgres Babelfish (a SQL Server-compatible Postgres flavor) and Timescaledb (time-series extension for Postgres) we can deliver open source time-series native capabilities on top of SQL Server protocol, available to SQL Server users.…

Data on Kubernetes Community

1
Why we created one more Operator for MySQL (DoK Day EU 2022) // Sergey Pronin 9:22

לפני 3 years9:22

9:22

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) There are plenty Kubernetes Operators for MySQL, including our own at Percona. In this session we will review existing Operators for MySQL and discuss why we are building a new one. Sergey is a passionate technology "driver". After graduation worked in various fields: internet service provider, financial sector and M&A business. Main focal points were infrastructure and products around it. At Percona as a Group Product Manager drives forward Kubernetes and Cloud databases solutions.…

Data on Kubernetes Community

1
Why run Postgres in Kubernetes (DoK Day EU 2022) // Gabriele Bartolini 10:02

לפני 3 years10:02

10:02

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Postgres should run inside your Kubernetes cluster. Yes, inside, not outside Kubernetes. After all, a database should be seen as an application, a special type of application - for which it is legitimate to require an additional level of care and attention. However, the small price you pay for this is worth the return on investment that your organization receives by running microservice applications, plus database combos, inside your Cloud Native infrastructure. In this session, I will cover why it is important to make such a decision. Discover the challenges and the opportunities that running Postgres inside Kubernetes presents. I will discuss what we all expect from a good Postgres operator, including self-healing, high availability, scalability, backup and recovery, performance, and - last but not least - security. I will then go beyond technical aspects to spark conversations about the holistic improvements that running a Cloud Native database brings to your organization. A long time open source programmer and entrepreneur, Gabriele has a degree in Statistics from the University of Florence. After having consistently contributed to the growth of 2ndQuadrant and its members through nurturing a lean and devops culture, he is now leading the Cloud Native initiative at EDB. Gabriele lives in Prato, a small but vibrant city located in the northern part of Tuscany, Italy - known for having hosted the first European PostgreSQL conferences. His second home is Melbourne, Australia, where he studied at Monash University and worked in the ICT sector. He loves playing the Blues with his Fender Stratocaster, but his major passions are called Elisabeth and Charlotte!…

Data on Kubernetes Community

1
What's New in Kubernetes Storage (DoK Day EU 2022) // Xing Yang 9:36

לפני 3 years9:36

9:36

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Kubernetes SIG Storage is responsible for ensuring storage is available for containers in a pod when the pod is scheduled on a node. There is the Container Storage Interface (CSI) for block and file storage that allows storage providers to write CSI drivers. There is also a COSI sub-project that is trying to add object storage support in Kubernetes. In this session, Xing will give an update on some of the features that SIG Storage is working on and discuss what might be coming in the future. Xing Yang is a Tech Lead in the Cloud Native Storage team at VMware. She is a co-chair of the CNCF TAG Storage, a co-chair of the Kubernetes SIG Storage, a co-chair of the Kubernetes Data Protection WG, and a maintainer in Kubernetes CSI. Before joining VMware, Xing was the Lead Architect of OpenSDS while working for Futurewei. She also worked at Dell EMC for many years and has developed deep expertise in storage, data protection, disaster recovery, cloud, and virtualization technologies.…

Data on Kubernetes Community

1
What we've learned from running a PostgreSQL managed service on Kubernetes (DoK Day EU 2022) // Oleksii Kliukin 11:06

לפני 3 years11:06

11:06

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Kubernetes is an emerging platform of choice for deploying and running PostgresSQL. Deploying 100 Postgres clusters is as easy as deploying one, and there is no need to tinker with tools like Ansible or Puppet. Resource sharing can be applied when it makes sense, allowing to run multiple Postgres databases in isolation on a single instance, each storing the data on a dedicated persistent volume. There are great open-source tools out there to deal with high-availability and backups than support or can be easily integrated into the Kubernetes workflow. Monitoring and alerting is easy to implement. People reported success in running Postgres on Kubernetes before. But there are also rough edges, like memory management or certain Postgres maintenance operations, such as installing extensions, that normally cause unnecessary database downtimes on Kubernetes. They are less of a problem for in-house deployments, but may become a deciding factor when running a managed service, competing with other such services running on bare-metal servers or virtual machines that are free of those issues. In this talk, I will share some of our learnings from running a managed PostgreSQL/TimescaleDB service on Kubernetes on AWS for a little more than a year: I’ll start with the motivation of running managed PostgreSQL on Kubernetes, the benefits and drawbacks. I’ll describe the architecture of the managed PostgreSQL cloud on Kubernetes I’ll zoom in on how we solved some of the Kubernetes-specific issues within our cloud, such as upgrading extensions without downtimes, taming the dreaded OOM killer, and doing regular maintenance and PostgreSQL major upgrades. I’ll share how open-source tools from the PostgreSQL ecosystem helps us to run the service and explain how we use them in a slightly non-trivial way. Oleksii has been working with PostgresSQL for almost 20 years, and has been deploying Postgres on Kubernetes since 2016, when his team at Zalando started the internal managed PostgreSQL service based on the in-house and open-source postgresql-operator. Around 2015, with some other team members, he stared working on a PostgreSQL HA project that later became Patroni. Long before that he was hacking PosgreSQL source code to implement binary replication on PostgreSQL 7.x, authoring some PostgreSQL extensions and contributing to the core PostgreSQL itself. He started PosgreSQL meetups in Berlin in 2015 and hopes to get back to meeting in-person somewhere in 2022. Being Ukrainian, he lives in Berlin for a bit more than 9 years with his wife, two children and numerous plants…

Data on Kubernetes Community

1
Weathering The Cloud Storm- Modern Data Management Patterns for Reliability and Availability (DoK Day EU 2022) // Denis Magda 10:46

לפני 3 years10:46

10:46

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) “Zero downtime” and “always-on” are illusions. All systems fail sooner or later, whether it’s a regional e-commerce website or a major cloud region hosting thousands of applications. That’s why, instead of chasing these illusions, it’s worth focusing on the nines of availability. Based on true stories, this session walks you through modern data availability and reliability patterns used by architects whose applications withstood major cloud outages. With the focus on the data storage layer and Kubernetes, you’ll learn: * How to architect the data layer in Kubernetes with the server, zone, and region-level resiliency in mind. * How to find a compromise between latency and availability for multi-region deployments. * How to ensure the data layer remains reliable (i.e., always returns expected data) even during a major incident. Denis Magda has spent half of his career working on distributed systems, applications, and databases. His experience spans from the development of distributed database engines and high-performance applications to training and education on the topic of distributed and cloud computing. Presently, Denis runs the Developer Relations team at Yugabyte and serves a PMC Member for Apache Ignite. He started his professional career at Sun Microsystems and Oracle, where he led one of the Java development groups and worked on technology evangelism efforts.…

Data on Kubernetes Community

1
Using Kubernetes to deliver a “serverless” service (DoK Day EU 2022) // Jim Walker 20:21

לפני 3 years20:21

20:21

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Serverless promises to change the way we consume software. It allows us to potentially pay for only that which we use and can help drive down operational costs to the minimal amount of resources necessary. Architecting for serverless requires a unique look at app logic and the way it is deployed. It takes a combination of the logical and physical worlds. An architectural pattern has emerged where we can scale ephemeral compute separate from services that need to persist. We use Kubernetes to deliver exactly this. A “serverless” experience that is driven and enabled by compute pods and storage pods. We also have used our experience running thousands of database clusters on Kubernetes to automate the operational expertise of managing a distributed database. In this talk, we will take a dive deep into the architecture of our application and share: * A definition and outline of the challenges of serverless * How we reworked our logic for a serverless approach * How we use Kubernetes to gain serverless autoscaling Jim is a recovering developer turned evangelist who loves useful, cool, cutting-edge tech. He loves to translate and distill complex concepts into compelling, more simple explanations that broader communities can consume. He is an advocate of the developer and an active participant in several open source communities.…

Data on Kubernetes Community

1
The many uses of Kubernetes cross cluster migration of persistent data (DoK Day EU 2022) // Ryan Kaw 7:39

לפני 3 years7:39

7:39

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Multiple clusters exist in most Kubernetes environments today, and number of clusters will increase overtime. The reasons for having multiple Kubernetes clusters are many, for example, overcoming scale limits, reducing complexity, geo separation, redundancy and having separate production, staging, and development environments. Once you have multiple K8S clusters, it can be useful to have the ability to easily move or duplicate workloads across these different clusters. Kubernetes does not have a native method to allow migration or duplication of workloads across clusters. Fortunately, there are tools that provide this functionality. In this presentation we will explore the different uses cases for cross cluster migration, and what is involved, and how these migration tools work. We'll cover some popular uses cases, such as, Disaster Recovery, Test/Dev, and performance testing. Migration could entail moving the entire cluster, or individual workloads. The components that need to be moved would include configuration and resources stored in etcd, and persistent data residing on PVCs. We'll cover the uses cases and challenges for migration, and run through an example of using one of these migration tools.…

Data on Kubernetes Community

1
The future of data on Kubernetes with Adobe and CNCF (DoK Day EU 2022) // Joseph Sandoval, Xing Yang & Sylvain Kalache 17:29

לפני 3 years17:29

17:29

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Some data-intensive workloads are easier to run in Kubernetes than others. Why? What needs to improve? Join us as we deep dive with Adobe and the CNCF about how easy (or not) it is to run different types of data workloads on Kubernetes – and what is being done both inside and outside of Kubernetes to make data workloads easier. Xing Yang is a Tech Lead in the Cloud Native Storage team at VMware. She is a co-chair of the CNCF TAG Storage, a co-chair of the Kubernetes SIG Storage, a co-chair of the Kubernetes Data Protection WG, and a maintainer in Kubernetes CSI. Before joining VMware, Xing was the Lead Architect of OpenSDS while working for Futurewei. She also worked at Dell EMC for many years and has developed deep expertise in storage, data protection, disaster recovery, cloud, and virtualization technologies. I find myself working at the intersection of OpenSource, Cloud Native software, SRE, and DevOps. My goal is to help bring a pragmatic opinion and shepherd organizations to stay focused on delivering value to their internal and external customers…

Data on Kubernetes Community

1
The Data on Kubernetes Landscape (DoK Day EU 2022) // Melissa Logan & Sylvain Kalache 10:25

לפני 3 years10:25

10:25

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) We know from the first Data on Kubernetes Report that 90% of respondents believe Kubernetes is ready for stateful workloads, but significant challenges remain. The DoK Community continues to grow and build a unique space where people share knowledge and have conversations that are shaping the next decade of data on Kubernetes. Melissa Logan and Sylvain Kalache will discuss the growth of DoK, DoKC, and introduce a new project to help end users on their journey to running data-intensive workloads on Kubernetes more easily and with better control.…

Data on Kubernetes Community

1
Testing the Mettle- Evaluating data solutions for large-scale production to check who stacks up (DoK Day EU 2022) // Dinesh Majrekar 9:26

לפני 3 years9:26

9:26

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) The state of the CNCF Storage options has exploded in the past few years, but if you had to choose a project to use today, how would you go about comparing each offering and choosing who to partner with for your future growth? Civo offers managed Kubernetes clusters powered by K3s to customers around the world. We manage thousands of Virtual Machines and stateful customer data within multiple data centres across several continents. In late 2021, Civo had the opportunity to evaluate the CNCF storage landscape to move to a new technology stack. Learn about the steps required to evaluate the market, testing vendor claims in the real world and what needs to be considered outside of the purely technological aspects of any product selection. In late 2021, Civo's Director of Innovation, Dinesh, spoke about the intricacies of migrating between two vendors (https://www.youtube.com/watch?v=GlUsPnSZI_Y&list=PLHgdNuGxrJt2-xlW_l2q1BAE3e4TsPnQo). Now join him as he talks about the wider project and some of business implications of hosting Data on Kubernetes Dinesh is Director of Innovation at Civo and has been the lead architect for the new Civo Stack Platform. Having worked in the hosting industry for many years, Dinesh has a passion for creating solutions that operate at scale. This not only applies to the technology stack, but for nurturing engineers through their career.…

Data on Kubernetes Community

1
Tech with project RapGOD (DoK Day EU 2022) // Abhijith Ganesh 8:50

לפני 3 years8:50

8:50

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) The Rap God project acts as a great entry point to many incoming open-source enthusiasts who are interested in learning about the cloud native ecosystem. The Rap-God project uses Kubernetes orchestration for a stateful case which is an emerging topic, the Rap God project acts as a demonstration of how to use such features of Kubernetes. The project will be using Stateful sets that’d deploy Apache Cassandra (for its first cycle) and eventually it’ll be implementing the same API endpoints for various databases that will be with/on Kubernetes. We in the community intend to do this with PersistenceVolumes and Persistent Volume Claims. Keeping in mind the issues, various developers face, we also will be making options for storage classes. The project will allow the members to explore how they can customize the whole storage class setup according to their setup. The project will be bringing Helm, Cassandra, Kubernetes and Argo under its watch and shall actively expand on its implementation with the further iterations. Abhijith Ganesh is an undergrad computer science major, currently pursuing his Freshman year. His areas of interest include DevOps, Kuberenetes and Open Source Projects. He is an active member of the DoK Community where he is currently an intern. He is also member of the Pyrsia and SeaQL communities.…

Data on Kubernetes Community

1
Serverless Event Streaming Applications as Functions on K8 (DoK Day EU 2022) // Timothy Spann 8:43

לפני 3 years8:43

8:43

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) We will walk through how to build serverless event streaming applications as functions running in a function mesh on kubernetes with cloud native messaging via Apache Pulsar. In this talk, you will deploy ML functions to transform real-time data on Kubernets. Tim Spann is a Developer Advocate @ StreamNative where he works with Apache Pulsar, Apache Flink, Apache NiFi, Apache MXNet, TensorFlow, Apache Spark, big data, the IoT, machine learning, and deep learning. Tim has over a decade of experience with the IoT, big data, distributed computing, streaming technologies, and Java programming. Previously, he was a Principal Field Engineer at Cloudera, a Senior Solutions Architect at AirisData and a senior field engineer at Pivotal. He blogs for DZone, where he is the Big Data Zone leader, and runs a popular meetup in Princeton on big data, the IoT, deep learning, streaming, NiFi, the blockchain, and Spark. Tim is a frequent speaker at conferences such as IoT Fusion, Strata, ApacheCon, Data Works Summit Berlin, DataWorks Summit Sydney, and Oracle Code NYC. He holds a BS and MS in computer science. https://www.datainmotion.dev/p/about-me.html https://dzone.com/users/297029/bunkertor.html https://conferences.oreilly.com/strata/strata-ny-2018/public/schedule/speaker/185963…

Data on Kubernetes Community

1
Running Kafka on Kubernetes, across three clouds at Adobe (DoK Day EU 2022) // Adi Muraru 16:48

לפני 3 years16:48

16:48

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Adobe runs dozens of Kafka clusters spread across both public (AWS and Azure) and private clouds to power the Adobe Experience Platform message bus. In this talk, we share the experience of running Kafka as a stateful Kubernetes workload in a cloud-native environment using the open-source Koperator and the tools we built to automate the whole lifecycle. While running a stateful service like Kafka on Kubernetes may be intimidating at the first glance, we share our thought process, the tools, and the results that can make this a reality in any organization. The Kubernetes Operator pattern helped us automate all the operational aspects for the lifecycle of the cluster; abstract away the cloud specifics allowing us to focus on Kafka; achieve increased resilience and elasticity; implement automated Kafka rebalancing using CruiseControl, and harness all the metrics to implement an observable environment. We also plan to demo how these all come together. Join us as we share our team's learnings on: - Considerations for running a stateful service in a multi-tenant Kubernetes cluster: - Pod sizing in terms of memory and CPU - Scheduling - Persistent Volume Claims (PVC) tuning - Resiliency topics - How to handle regular Ethos upgrades and maintenance - How to bring custom monitoring solutions - Advanced network topics such as using dedicated VIPs If you or your team ever wanted to learn how to run your own stateful service in Kubernetes and/or see what's possible, come join us! Computer Science Ph.D. and Principal Scientist at Adobe having experience in complex, large scale distributed systems. My focus currently is on designing and building the message streaming platform for Adobe Experience Platform using Apache Kafka. The endeavor is unique as it spans both Amazon AWS and Microsoft Azure clouds and handles billions of messages a day.…

Data on Kubernetes Community

1
Running a database on local NVMes on Kubernetes (DoK Day EU 2022) // Tomáš Nožička & Maciej Zimnoch 9:42

לפני 3 years9:42

9:42

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Running a database on Kubernetes with persistent storage is relatively easy but when it comes to performance it won’t match local NVMes. This talk will show you how to set up the local NVMes for Kubernetes, how to handle the application and cluster lifecycle in a safe manner and share our experience with running ScyllaDB with local NVMes on different Kubernetes cloud providers. Tomas leads the development of Scylla Operator (https://github.com/scylladb/scylla-operator), a Kubernetes operator to manage ScyllaDB. Previously, he worked on a self-hosted, auto-upgrading Kubernetes control plane for RedHat OpenShift. Tomas is an Emeritus Kubernetes SIG-Apps approver. Maciej is a Go and C++ enthusiast. He is a software engineer working on ScyllaDB management tools. Previously he worked in network companies where he delivered multiple features to SDN solutions and LTE networks.…

Data on Kubernetes Community

1
Resilient Redis (DoK Day EU 2022) // Hrittik Roy & Ryan Gray 7:45

לפני 3 years7:45

7:45

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Redis is a widely used open-source in-memory data store and cache that has become a key component in the development of scalable microservice systems. While all of the main cloud providers provide fully managed Redis services (Amazon ElastiCache, Azure Cache for Redis, and GCP Memorystore), it may also be simply implemented in Kubernetes if you require additional control over the Redis configurations. This session will lead you through setting up a Redis cluster on Kubernetes with data persistence. The lesson would concentrate on establishing a "prod"-ready Redis cluster that considers storage and persistence even after restarts and discusses recommended practices and considerations. With the help of monitoring and hardware optimizations for your cluster, you'll learn about measuring your performance and best practices for Redis on Kubernetes. We would conclude whether operators or Helm chart will be a more mature option for your Data on Kubernetes needs with Redis and a few tips as a production checklist. Ryan Gray is a Technical Consultant with Portainer.io. Before that he spent a couple of decades building ISPs, carrier Ethernet networks, data centers, and software. He is currently obsessed with container orchestration, GitOps, edge computing, and why naming things is so hard. Hrittik is a Community Advocate at Portainer and a student in his sophomore year. Hrittik focuses on building tech communities and software with the help of cloud native technologies. He has been a speaker at conferences like the Cloud Summit and conducted various events that have collectively empowered students and professionals on DevOps and Cloud.…

Data on Kubernetes Community

1
PV TrashCan - Protection against accidental deletion of PVs or Namespaces (DoK Day EU 2022) // Veda Talakad, Aditya Kulkarni & Aditya Dani 11:07

לפני 3 years11:07

11:07

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Accidental PVC delete or namespace delete can cause the Persistent Volume to get deleted. Such volumes lose their data and the stateful applications lose their state. By the use of Persistent Volume TrashCan, users can get a grace period to undo such unintended delete operation. The deleted Persistent Volumes are staged for delayed deletes. They continue to live even after being deleted from k8 perspective, for a configurable time(retention period) and based on the system’s usage. The storage class of the PVC can dictate if they need to be staged for a delayed delete. StorageClass can also allow for configurable retention period. To recover a deleted PersistentVolume, users can create a new namespace with the same name and reapply the original PVC spec. The PVC will reference a special StorageClass to indicate that the new PersistentVolume needs to be restored from the TrashCan. This will allow the application to restart with the right state and data. This talk will showcase how to overcome one of the admin’s pain point seen in field involving accidental deletions of PVCs by using advanced storage management solutions in Kubernetes. Veda Talakad is a Software professional with BS in electronics and communications mostly working in storage domain. Some of the professional areas of interests include scalability in distributed environment and cloud-native technologies for data management, Aditya Kulkarni: I have 10 years of experiance in Enterprise Data management. I have worked on All Flash Array at Netapp from the device driver layer to WAFL Filesystem. Next, I worked at Portworx where I developed and enhanced the number one data platform for Kubernetes. Then I moved to Pure Storage as part of Portworx acquisition. Aditya Dani is an architect at Portworx by PureStorage, that enables users to run any cloud-native data service, in any cloud, using any Kubernetes platform. He is one of the contributors to the open source project Stork that provides storage awareness to the Kubernetes scheduler. Prior to Portworx, Aditya worked for Amazon on their Music Recommendations Engine. He enjoys working on distributed systems and loves programming in Go.…

Data on Kubernetes Community

1
Protecting data with CSI Volume Snapshots on Kubernetes (DoK Day EU 2022) // Grant Griffiths 11:10

לפני 3 years11:10

11:10

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) The container storage interface (CSI) is a contract between different container orchestrators (Kubernetes, Nomad, etc) and storage plugins. This contract is a set of gRPC services for provisioning, utilizing, and snapshotting storage volumes. In this talk, we will focus on one aspect of the CSI spec: Volume Snapshots. The talk will cover volume snapshots on Kubernetes CSI, with a deep dive on Kubernetes CSI snapshots. A short demo on the basics of the CSI snapshots on Kubernetes will help show how easy it is to start using CSI snapshots. After covering the basics on both container schedulers, we’ll deep dive into how backup systems can programmatically interact with the Kubernetes CSI snapshots client. This example will be based on Stork, an open-source backup system that has implemented the CSI spec. I believe this talk will be beneficial to the community because it covers some storage-related topics that have not been covered much in previous conferences. I’m hoping to bring the unique perspective of a Kubernetes CSI and Nomad CSI contributor, while also covering the CSI plugin maintainer and end-user perspective. With this wide audience, many conference attendees will be able to learn about stateful applications, CSI volume snapshots, and developing CSI drivers for any container scheduler. Grant Griffiths is a core contributor and reviewer for the Kubernetes CSI and SIG Storage communities working at Portworx. For kubernetes-CSI, he helped bring the external-snapshotter to Beta in Kubernetes 1.17 and GA in Kubernetes 1.20, improved support for secrets, release tooling, and fixed various snapshot controller bugs. At Portworx, he works on control plane components such as the CSI Driver, SDK, CLI, and security. Previously, Grant was a Senior Software Engineer at GE Digital where he built Data Services in Go on k8s. He has given talks at various community conferences: Two talks at GopherCon London 2018 https://youtu.be/gm34Aph9v0M https://youtu.be/MAopsd6kdY0 KubeCon Europe 2020: https://youtu.be/Yt5wpZ6raDA HashiTalks: Deploy 2021: https://youtu.be/PXVatM1xZxg…

Data on Kubernetes Community

1
Operator Lifecycle Management (DoK Day EU 2022) // Julian Fischer 15:21

לפני 3 years15:21

15:21

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) The ability to extend Kubernetes with Custom Resource Definitions and respective controllers has led to the OperatorSDK, which became the de facto standard for data service automation on Kubernetes. There are countless operator implementations available, and new operators are being released on a daily basis. Organizations managing hundreds of Kubernetes clusters for dozens of developer teams are also challenged to manage the lifecycle of hundreds of Kubernetes operators. The goal is to keep the operational overhead to a minimum. In this talk, a closer look into the lifecycle of operators will be presented. With an understanding of how operators evolve, it becomes clear what challenges during operator upgrades. A brief overview of lifecycle management tools such as Helm, OLM, and Carvel is presented in this context. In particular, it will be discussed whether these tools can help, which restrictions apply and where further development would be desirable. At the end of this talk, you will know what operator lifecycle management is about, what its challenges are, and which tools may be used to reduce operational friction. Julian Fischer, CEO of anynines, has dedicated his career to the automation of software operations. In more than fifteen years, he has built several application platforms. He has been using Kubernetes, Cloud Foundry, and BOSH in recent years. Within platform automation, Julian has a strong focus on data service automation at scale.…

Data on Kubernetes Community

1
Operating FoundationDB on Kubernetes (DoK Day EU 2022) // Johannes M. Scheuermann 8:56

לפני 3 years8:56

8:56

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) FoundationDB is an open-source distributed transactional Key-Value store that is used by multiple companies like Apple, Snowflake and VMWare Tanzu (previously Wavefront). This talk will cover the design of the FoundationDB operator and lessons learned from operating FoundationDB on Kubernetes. We will discuss some of the missing pieces in Kubernetes to make it easier to operate FoundationDB on top of it and how we solved those challenges in the operator. We will focus on the pieces of the FoundationDB operator that are different to most other operators and why we decided to implement those pieces like they are. We will also discuss how to run an high available FoundationDB cluster on top of Kubernetes and what different choices a user has. We will also cover some challenges that arise when running stateful services at scale on top of Kubernetes and how they can be managed. At the end of this talk we will give an outlook for future design changes and planned features in our operator. The main take-away from this talk is to understand how to run and operate FoundationDB on Kubernetes. Johannes started his journey in the Kubernetes eco-system in early 2015, onboarding projects and applications onto Kubernetes. Since 2020 Johannes works as an SRE for FoundationDB at Apple and is co-leading the development of the open source FoundationDB operator.…

Data on Kubernetes Community

1
One Click to Run Apache Spark as a Service on Kubernetes (DoK Day EU 2022) // Bo Yang 9:38

לפני 3 years9:38

9:38

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) It is still challenging to run Apache Spark and other big data processing workload on Kubernetes, especially in large scale. People need to address various issues like resource isolation, queuing, and cost efficiency. This session will share details about those challenges and how to address them. We will also present a convenient (one-click) way to deploy Apache Spark on Kubernetes, and dramatically lower the barrier to use Spark. Bo has been working in big data area for 10 years, and gained intensive experience of running Apache Spark in large scale while he worked in in Uber. In the Kubernetes era, he focuses on building world class Apache Spark Service, and making the tooling available to the Spark and Kubernetes communities.…

Data on Kubernetes Community

1
Microservices and Kubernetes for your Full Data Lifecycle (DoK Day EU 2022) // Steve Pousty 14:26

לפני 3 years14:26

14:26

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Data doesn’t magically appear in our data centers. There are usually several phases and several storage locations along its journey throughout your organization. New architectural patterns, such as microservices, and new technology, such as Kubernetes are changing how we can think about and manage the large volumes of data coming at us. In this talk we will begin by quickly introducing the architecture and technology and how they make our lives better. From there it’s live demo time combining Java microservices, a processing caching service, a messaging layer, and a relational database all running in Kubernetes . This application will be handling frequently updated data, generating alerts on specific data events, and simultaneously populating a system of record. Come for the discussion, hands on demo, and witty banter! Steve is a Dad, Partner, Son, and Senior Developer Readiness Engineer for VMware Tanzu. In addition to showing off all the great work of the Tanzu team, he helps drive Tanzu developer experience. He can teach you about Containers, Kubernetes, Data Analysis, Java, Python, PostgreSQL, Microservices, and some JavaScript. He has deep subject area expertise in GIS/Spatial, Statistics, and Ecology. Before Tanzu, Steve was a developer Advocate for Crunchy Data, DigitalGlobe, Red Hat, LinkedIn, deCarta, and ESRI. Steve has a Ph.D. in Ecology and can easily be bribed with offers of bird watching or fly fishing. He has spoken at hundreds of conferences and done over 100 workshops including Monktoberfest, Red Hat Summit, MongoNY, JavaOne, FOSS4G, ODSC, AjaxWorld, GeoWeb, Where2.0, and OSCON.…

Data on Kubernetes Community

1
Leveraging Running Stateful Workloads on Kubernetes for the Benefit of Developers (DoK Day EU 2022) // Arsh Sharma, Lapo Elisacci & Ramiro Berrelleza 14:02

לפני 3 years14:02

14:02

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Kubernetes comes with a lot of useful features like Volumes and StatefulSets, which make running stateful workloads simple. Interestingly, when combined with the right tools, these features can make Kubernetes very valuable for developers wanting to run massive production databases in development! This is exactly what was seen at "Extendi". The developers at Extendi deal with a large amount of data in their production Kubernetes clusters. But when developing locally, they didn't have an easy way of replicating this data. This replication was needed because it allowed developers to test new features instantaneously without worrying if they would work as expected when pushed to production. But replicating a 100Gb+ production database for development wasn't turning out to be an easy task! This is where leveraging Kubernetes + remote development environments came to the rescue. Running data on Kubernetes turned out to be way faster than any of the traditional approaches because of Kubernetes' ability to handle stateful workloads exceptionally well. And since Extendi already used Kubernetes in production - the setup process was fairly simple. This talk will cover practical steps on how leveraging Kubernetes based development environments allowed dev teams at Extendi to run production data on Kubernetes during development using features like Volume Snapshots, having a huge positive impact on developer productivity. Arsh is a Developer Experience Engineer at Okteto. He is an active contributor to the upstream Kubernetes project and was awarded the Kubernetes Contributor Award for his contributions in 2021. Arsh has written blogs and spoken about different topics in the cloud-native ecosystem at various conferences before, including KubeCon + CloudNativeCon + Open Source Summit China 2021. He has also been on the Kubernetes Release Team since the 1.23 release. He also serves as the New Contributor Ambassador for the Documentation Special Interest Group of the Kubernetes project and continuously mentors new folks in the community. Previously, he worked at VMware and was an active contributor to other CNCF projects, including cert-manager and Kyverno. Lapo is a Software Engineer currently leading the development team of a Social Listening and Audience Intelligence platform. He started coding at the early age of 14 and since he turned his passion into a real job, he has always been looking for boosting his knowledge by constantly researching for newer and newer technologies. Active on Ruby Open Source projects Ramiro Berrelleza is one of the founders of Okteto. He has spent most of his career (and his free time) building cloud services and developer tools. Before starting Okteto, Ramiro was an Architect at Atlassian and a Software Engineer at Microsoft Azure. Originally from Mexico, he currently lives in San Francisco.…

Data on Kubernetes Community

1
Kanister & Kopia - An Open-Source Data Protection Match Made in Heaven (DoK Day EU 2022) // Pavan Navarathna 13:38

לפני 3 years13:38

13:38

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Cloud-native applications comprise various components, including data services, storage systems, and related Kubernetes objects. Each component requires its own data protection tools, strategy, and domain expertise. A robust solution aligned with business requirements often involves complex workflows. What if there was a way to coordinate the implementation of these workflows while optimizing how backups are moved into storage? During this talk, Pavan will demonstrate how two open-source tools, Kanister and Kopia, work together to optimize backup and recovery for Kubernetes applications. - Kanister allows domain experts to capture application-specific workflows in Blueprints to extend and share. - Kopia uses state-of-the-art encryption algorithms (AES-256 or ChaCha20) and compresses data to save bandwidth and storage. This talk is for anyone interested in running stateful workloads in production on Kubernetes. Attendees will leave armed with a streamlined way to protect Kubernetes applications. Pavan joined Kasten by Veeam in March 2018, where he manages a team of cloud-native engineers developing creative solutions for managing and protecting stateful application data in Kubernetes. Prior to Kasten, he was a part of NetApp’s Cloud Data Services Group, where he worked on the data protection stack for a cloud-integrated backup appliance. Pavan is an active contributor to open-source projects including Kanister, a framework for application-level data management on Kubernetes; and Kopia, a fast and secure backup tool. He holds a Master's degree in Computer Science from the University of Florida.…

Data on Kubernetes Community

1
Is your database in Kubernetes production ready (DoK Day EU 2022) // Mykola Marzhan 15:21

לפני 3 years15:21

15:21

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) It only looks simple to run databases in Kubernetes. In fact, it is too many things needed to be considered before running any database in Kubernetes. Failover and traffic switching, replication and data consistency/loss after failover, upgrades, DB and node-level configuration, CNI, backups, monitoring, etc. After this talk, you will have a complete list of questions that should be checked before running a database in production. Mykola has been shorting “Time-to-market” in software companies for more than ten years. Most of his career he has been focused on the *development* of monitoring, update and deployment systems.…

Data on Kubernetes Community

1
How to protect your data (DoK Day EU 2022) // Sarah Julia Kriesch 6:39

לפני 3 years6:39

6:39

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) How can you keep your data secure and how can you transfer them on a secure way? You will learn to encrypt your data, that you can use them Kubernetes based in a Multi Cloud environment. Sarah Julia Kriesch is working as a DevOps Consultant for Accenture in Germany. She has studied Computer Science at the TH Nürnberg and worked as a Linux System Administrator/ System Engineer for an ISP and in the area of Cloud Computing before. She is also contributing to Open Source since around 10 years.…

Data on Kubernetes Community

1
Growing up fast - Kubernetes and Real-Time Analytic Applications (DoK Day EU 2022) // Robert Hodges 15:30

לפני 3 years15:30

15:30

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Kubernetes is turning into a preferred platform for real-time analytic app that crunch billions of events per day and return insights in seconds. In this talk we'll introduce the standard analytic app design pattern of fast event streams coupled with low-latency data warehouses, using open source projects. We'll then walk through deploying the pipeline on Kubernetes from ingest to end user access. We'll touch on use of operators, scaling, monitoring, upgrade, security, and approaches to adding custom components. Attendees can expect to leave with concrete lessons about how to stand up low-latency analytics quickly on Kubernetes. Robert Hodges is CEO of Altinity, an enterprise provider for ClickHouse data warehouse. He's also a database geek with experience on at least 20 DBMS types. Robert caught the Kubernetes bug at VMware in 2018.…

Data on Kubernetes Community

1
Graph in Kubernetes Panel (DoK Day EU 2022) // Wey Gu, Cheukting Ho & Feynman Zhou 20:15

לפני 3 years20:15

20:15

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Graph databases are the fastest growing data store in the world. According to Gartner, the application of graph processing and graph DBMSs will grow at 100 percent annually through 2022 to continuously accelerate data preparation and enable more complex and adaptive data science. However, it is often difficult for data and analytics professionals to distinguish between different implementation models, and to fit them to their use case. This panel will speak directly to Kubetnetes users and provide them with the context they need to run stateful workloads. Topics for discussion would be: - graph databases on Kubenetes challenges, lessons learnt, and patterns for deployment - graph uses case on k8s, including scaling knowledge graphs - K8s makes graph better - technical patterns to use graph databases to get the maximum out of k8s - the panels’ learnings from running graph data on Kubernetes…

Data on Kubernetes Community

1
From Laptop to Cloud. Developing Cloud-Native Applications with Containerized Databases (DoK Day EU 2022) - Nic Vermandé 17:16

לפני 3 years17:16

17:16

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) With the advent of microservices in Kubernetes, individual developer teams now manage their own data, middleware, and databases. Automated tests and CI/CD pipelines have to be revisited to include these new requirements. This session will discuss and demonstrate how to use Kustomize and Tekton to provide Kube-Native automated workflows taking into account new parameters such as database operators, StorageClass and PVC. In this talk, Nic is also going to provide some insights on how to optimize Tekton to work with multiple workspaces and overcome some of the affinity limitations. The demonstration will focus on building a comics cards web application using a flask-based frontend and leveraging postgreSQL as the database. It will cover the automation of multiple lifecycle stages: - local laptop testing automation on K3S with Kustomize - production deployment using Flux, Tekton and the Zalando PostreSQL operator Nicolas is an experienced hands-on technologist, evangelist, and product owner who has been working in the fields of Cloud-Native technologies, Open Source Software, Virtualization, and Datacenter networking for the past 17 years. Passionate about enabling users and building cool tech solving real-life problems, you'll often see him speaking at global tech conferences and online events, spreading the word, and walking the walk with customers and users.…

Data on Kubernetes Community

1
Disaggregated Container Attached Storage - Yet Another Topology with What Purpose (DoK Day EU 2022) // Nick Connolly 9:32

לפני 3 years9:32

9:32

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) The storage topology in vogue seems to cycle every few years. Internal storage is followed by centralized Storage Area Networks only to be superseded by one-size-fits-all Hyperconverged models - until scalability constraints led to distributed storage. Then comes NVMe, offering blistering speeds that all of these storage stacks struggle with. Kubernetes inspires Container Attached Storage aspiring to be the perfect model, so why is disaggregated storage now making an appearance? This talk considers the motivations behind yet another storage topology and examines a modern, flexible architecture for delivering high-performance storage under Kubernetes. Nick Connolly is a pioneer of storage virtualisation and the Chief Scientist at DataCore, where his background in real-time computing and multiprocessing led to the creation of a world-class high-performance storage stack on Windows. He holds patents ranging from highly scalable algorithms through to data protection techniques. Recently he has been working with OpenEBS to bring the power and performance of NVMe to Kubernetes.…

Data on Kubernetes Community

1
Datashim - a framework for declarative management of datasets on Kubernetes (DoK Day EU 2022) // Srikumar Venugopal 15:36

לפני 3 years15:36

15:36

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Many ML pipelines depend on shared filesystems for input, output and intermediate data storage. Standards such as CSI have made it possible for applications in Kubernetes to access a variety of data storage systems. Yet, data scientists still have to deal with low-level details of data access in order to execute their pipelines in Kubernetes. Datashim is a framework that manages the lifecycle of a Dataset object, a CustomResourceDefinition that represents a source of data. Datashim takes care of the details of data access while Kubernetes pods can declaratively access the data by referencing a Dataset in their specifications. This talk will describe Datashim and the Dataset object, discuss its use in ML pipelines, and demonstrate how its pluggable architecture is designed for the development of caching, scheduling and governance plugins. Datashim is an incubating project of the Linux Foundation Data and AI Foundation Srikumar Venugopal is a Research Scientist in IBM Research Europe in Dublin, Ireland. His research interests lie in the area of cloud computing and large-scale distributed systems, specifically in the topics of middleware, resource management, and scalability. He is the co-founder and current lead for the Datashim project.…

Data on Kubernetes Community

1
Build your own social media analytics with Apache Kafka (DoK Day EU 2022) // Jakub Scholz 10:22

לפני 3 years10:22

10:22

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Apache Kafka is more than just a messaging broker. It has a rich ecosystem of different components. There are connectors for importing and exporting data, different stream processing libraries, schema registries and a lot more. This talk will show how to use it to read data from social networks such as Twitter, process them and use machine learning to analyze them. And all of it will be of course running on top of Kubernetes. Jakub works at Red Hat as Senior Principal Software Engineer. He has long-term experience with messaging and currently focuses mainly on Apache Kafka and its integration with Kubernetes. He is one of the maintainers of the Strimzi project which provides tooling for running Apache Kafka on Kubernetes. Before joining Red Hat he worked as messaging and solution architect in the financial industry.…

Data on Kubernetes Community

1
Bringing Apache Cassandra closer to Kubernetes (DoK Day EU 2022) // Jake Luciani 9:41

לפני 3 years9:41

9:41

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) What does Kubernetes provide that allows us to reduce the complexity of Apache Cassandra while making it better suited for cloud native deployments? That was the question we started with as we began a mission to bring Cassandra closer to Kubernetes and eliminate the redundancy. Many great open source databases have been adapted to run on Kubernetes, without relying on the deep ecosystem of projects that it takes to run in Kubernetes(there is a difference). This talk will discuss the design and implementation of the Astra Serverless Database which re-architected Apache Cassandra to run only on Kubernetes infrastructure. Built to be optimized for multi-tenancy and auto-scaling, we set out with a design goal to completely separate compute and storage. Decoupling different aspects of Cassandra into scaleable services and relying on the benefits of Kubernetes and it's ecosystem created a simpler more powerful database service than a stand alone, bare-metal Cassandra cluster. The entire system is now built on Apache Cassandra, Stargate, Etcd, Prometheus, and object-storage like Minio or Ceph. In this talk we will discuss the downstream changes coming to several open source projects based on the work we have done. Jake is a lead developer and software architect at DataStax with over 20 years of experience in the areas of distributed systems, finance, and manufacturing. He is a member of the Apache Foundation and is on the project committee of the Apache Cassandra, Arrow, and Thrift projects. Jake has a reputation for developing creative solutions to solve difficult problems and fostering a culture of trust and innovation. He believes the best software is built by small diverse teams who are encouraged to think freely. Jake received his B.S. in Computer Science from Lehigh University along with a minor in Cognitive Science.…

Data on Kubernetes Community

1
Autoscaling Stateful Workloads in Kubernetes (DoK Day EU 2022) // Mohammad Fahim Abrar & Md. Kamol Hasan 10:14

לפני 3 years10:14

10:14

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 (https://youtu.be/Xi-h4XNd5tE) Managing stateful workloads in a containerized environment has always been a concern. However, as Kubernetes developed, the whole community worked hard to bring stateful workloads to meet the needs of their enterprise users. As a result, Kubernetes introduced StatefulSets which supports stateful workloads since Kubernetes version 1.9. Users of Kubernetes now can use stateful applications like databases, AI workloads, and big data. Kubernetes support for stateful workloads comes in the form of StatefulSets. And as we all know, Kubernetes lets us automate many administration tasks along with provisioning and scaling. Rather than manually allocating resources, we can generate automated procedures that save time, it lets us respond faster when peaks in demand, and reduce costs by scaling this down when resources are not required. So, it’s really important to capture autoscaling in terms of stateful workloads in Kubernetes for better fault tolerance, high availability, and cost management. There are still a few challenges regarding Autoscaling Stateful Workloads in Kubernetes. They are related to horizontal/vertical scaling and automating the scaling process. In Horizontal Scaling when we are scaling up the workloads, we need to make sure that the infant workloads join the existing workloads in terms of collaboration, integration, load-sharing, etc. And make sure that no data is lost, also the ongoing tasks have to be completed/transferred/aborted while scaling down the workloads. If the workloads are in primary-standby architecture, we need to make sure that scale-up or scale-down happens on standby workloads first, so that the failovers are minimized. While scaling down some workloads, we also need to ensure that the targeted workloads are excluded from the voting to prevent quorum loss. Similarly, while scaling up some workloads, we need to ensure that new workloads join the voting. When new resources are required, we have to make the tradeoff between vertical scaling and horizontal scaling. And when it comes to Automation, we have to determine how to generate resource (CPU/memory) recommendations for the workloads. Also, when to trigger the autoscaling? Let’s say, a group of workloads may need to be autoscaled together. For example, In sharded databases, each shard is represented by one StatefulSet. But, all the shards are treated similarly by the database operator. Each shard may have its own recommendations. So, we have to find a way to scale them with the same recommendations. Also, we need to determine what happens when an autoscaling operation fails and what will happen to the future recommendations after the failure? There can be some workloads that may need a managed restart. For example, in a database, secondary nodes may need to be restarted before the primary. In this case, how to do a managed restart while autoscaling? Also, we need to figure out what happens when the workloads are going through maintenance? We will try to answer some of those questions throughout our session. ----- Fahim is a Software Engineer, working at AppsCode Inc. He has been involved with Kubernetes project since 2018 and is very enthusiastic about Kubernetes and open source in general. ----- MD Kamol Hasan is a Professional Software Developer with expertise in Kubernetes and backend development in Go. One of the lead engineers of KubeDB and KubeVault projects. Competitive contest programmer participated in different national and international programming contests including ACM ICPC, NCPC, etc…

Data on Kubernetes Community

1
1000 node Cassandra cluster on Amazon's EKS (DoK Day EU 2022) // Matt Overstreet 7:19

לפני 3 years7:19

7:19

https://go.dok.community/slack https://dok.community/ From the DoK Day EU 2022 ( https://youtu.be/Xi-h4XNd5tE ) Come here about our experience scaling Cassandra on EKS to over 1000 nodes and 20 million transactions per second. This session will cover the lessons learned, successes, failures, and tools used to get there. Usability is Matt’s mission. He has worked with Federal, Fortune 500, and small businesses to help collect, mine and interact with data. When solving a problem, Mr. Overstreet synthesizes experience from a liberal arts and technical background. Matt has previously presented community webinars for DataStax and spoken at the search focused Haystack conference.…

Data on Kubernetes Community

1
Dok Talks #131 - How to win friends and influence businesses // Fabian Met 1:00:48

לפני 3 years1:00:48

1:00:48

https://go.dok.community/slack https://dok.community ABSTRACT OF THE TALK In this talk I share my personal experience where when I was working for a client the company had a hard time innovating and did not have a lot of faith in their software/infrastructure engineers. From a 'distance' I saw a lot of potential where they already had knowledgable engineers but lacked the experience to get started as a company but also as individual engineers. I created a community at that client, filled with their engineers. It was open to everyone who wanted to learn and we saw everyone's input as equal. Because of that safe space the company now runs almost all their IT in the cloud in a way that made both the engineers and the managers happy. All simply by just being a nice guy! BIO Hi my name is Fabian Met and I have been working in the cloud native space for the past couple of years. I love to learn new tech and do so almost every day! When I learn something new I then always use that knowledge to teach people how tech that might seem difficult at first can actually be quite manageable! Making sure that people do not experience these huge hills they sometimes face trying to learn something new is what I love to do! My role as Techfluencer means that I will help companies and people alike to see the benefit and required investment to start innovating and using new technology. I don't do this as an authoritative figure, instead I love to talk to people and hear their stories and challenges. Listening to them and talking to them usually leads to insight for both parties and possible solutions that previously might not have seen feasible! KEY TAKE-AWAYS FROM THE TALK Even though you might not be in a position of power, I will show you that no matter who you are, be that a manager, engineer or even the janitor you can help make positive change in your company!…

Data on Kubernetes Community

1
Dok Talks #130- Leaning on Kubernetes Portability to Manage Databases Anywhere // Robert Hodges 1:04:45

לפני 3 years1:04:45

1:04:45

https://go.dok.community/slack https://dok.community ABSTRACT OF THE TALK What if databases only ran in a single place? That would be useless. But it's what we get with most database-as-a-service offerings in the public cloud. Kubernetes has the potential to change the game and let users run complex database anywhere they want. This talk tells the story of how we built a portable ClickHouse cloud service on Kubernetes. It runs on AWS and GCP, of course. Even better, we're working on managing databases in *any* Kubernetes cluster, even those behind firewalls and NAT gateways. We can pull it off because Kubernetes is truly portable. We'll share what we've learned about using Kubernetes portability create a new breed of managed database. We'll speculate about what this portends for the future of database management. BIO Robert Hodges has worked with over 20 different DBMS types starting with M204 back in 1983. His current favorite is ClickHouse, a popular open source data warehouse. He caught the Kubernetes bug at VMware in 2018 and has been working on it ever since. In his day job he runs Altinity, an enterprise software and services provider for ClickHouse data warehouses. KEY TAKE-AWAYS FROM THE TALK Kubernetes is a great data platform even for complex, stateful systems like data warehouses. This talk shows how Kubernetes' portability is not just a convenience. It is a powerful tool that allows users to locate data anywhere it's convenient but still have a SaaS user experience for administrators.…

Data on Kubernetes Community

1
Dok Talks #129 - Databases Operations and the Cloud // Barak Nissim 51:20

לפני 3 years51:20

51:20

https://go.dok.community/slack https://dok.community ABSTRACT OF THE TALK This session walks through the basics of how data is represented in Kubernetes, from the grounds up and explores how databases - as a top consumers of data in this space - are handling multi-node and multi-cluster operations. we will review how operators help and why having a data layer is important when building a kubernetes-native application. BIO With over 15 years of IT experience from core datacenter infrastructure to advanced cloud technologies, Barak leads ionir’s product and solutions portfolio, engaging with customers, partners, and acts as kubernetes-native data advocate. KEY TAKE-AWAYS FROM THE TALK - Running Data on Kubernetes is fun and easy - Running business critical workloads as Databases require a data layer to improve availability, scale and bursting. - Multi cloud with Data is important for bursting and migrations…

Data on Kubernetes Community

1
Dok Talks #126- Automatically Instrument Kubernetes Apps with OpenTelemetry // James Blackwood-Sewell 1:03:40

לפני 3 years1:03:40

1:03:40

https://go.dok.community/slack https://dok.community ABSTRACT OF THE TALK The rise of Kubernetes has triggered an exponential growth of metric and trace data. This talk explores capturing and persisting telemetry data from Kubernetes apps without any specific code changes using the OpenTelemetry Operator. BIO James Blackwood-Sewell is currently a Principal Developer Advocate at TimescaleDB in Sydney. Before that he meandered through the PostgreSQL, Kubernetes and Prometheus landscapes as the Chief Architect at Jirotech. KEY TAKE-AWAYS FROM THE TALK Storing observability data from Kubernetes on Kubernetes doesn't need to be hard - sometimes you don't even need to change your code to do it!…

Data on Kubernetes Community

1
Dok Talks #128- Getting Started with the Kubernetes Secrets Store CSI Driver // Kim Schlesinger 53:10

לפני 3 years53:10

53:10

https://go.dok.community/slack https://dok.community ABSTRACT OF THE TALK In Kubernetes, it can be difficult to keep application API keys, access tokens and passwords safe. There are several different approaches to solving this problem, and in this talk Kim will demonstrate how to install Hashicorp Vault and the Secrets Store CSI Driver so that your applications can access secrets stored in ephemeral volumes. BIO Kim is technologist and educator, and she currently works as a Developer Advocate at DigitalOcean. Kim lives in Denver, Colorado in the USA. KEY TAKE-AWAYS FROM THE TALK How the Kubernetes Secrets Store CSI Driver with Vault uses Kubernetes volumes…

Data on Kubernetes Community

1
Dok Talks #127 - Flux for Helm Users! // Scott Rigby 1:21:35

לפני 3 years1:21:35

1:21:35

https://go.dok.community/slack https://dok.community ABSTRACT OF THE TALK Welcome Helm users! CNCF Flux has a best-in-class way to use Helm according to GitOps principles. For you, that means improved security, reliability, and velocity - no more being on the pager on the weekends or having painful troubleshooting or rollback when things go wrong. Built on Kubernetes controller-runtime, Flux’s Helm Controller is an example of a mature software agent that uses Helm’s SDK to full effect. Flux’s biggest addition to Helm is a structured declaration layer for your releases that automatically gets reconciled to your cluster based on your configured rules: ⭐️ The Helm client commands let you imperatively do things ⭐️ Flux Helm Custom Resources let you declare what you want the Helm SDK to do automatically. In addition, Scott will show how to use Helm Charts to run reliable stateful workloads. If you want to follow along with Scott’s demo, here are a couple of resources to help you prepare ahead of time: 📄 Flux for Helm Users Docs: https://fluxcd.io/docs/use-cases/helm/ 📄 Flux Guide: Manage Helm Releases: https://fluxcd.io/docs/guides/helmrel... BIO Scott is a Brooklyn based interdisciplinary artist and Developer Advocate at Weaveworks. He co-founded the Basekamp art and research group in 1998 and the massively collaborative Plausible Artworlds international network. In technology he enjoys helping develop open source software that anyone can use, most recently projects in the cloud native landscape including co-maintaining Helm and Flux. In daily decisions, large or small, he tries to help make the world a better place for everyone. KEY TAKE-AWAYS FROM THE TALK Scott will take you on a tour of Flux’s Helm Controller, share the additional benefits Flux adds to Helm and then walk through a live demo of how to manage helm releases using Flux.…

Data on Kubernetes Community

1
Dok Talks #125- Mission and Vision of the Rap-God-Project // Abhijith Ganesh 18:55

לפני 3 years18:55

18:55

https://go.dok.community/slack https://dok.community ABSTRACT OF THE TALK Explanation of how the how-to-dok project evolved into rap-god-api BIO An aspiring tech enthusiast who's massively into Data Science, Kubernetes and Operations(DevOps)

Data on Kubernetes Community

1
Dok Talks #124 - Intro to Druid on Kubernetes // Sergio Ferragut 54:40

לפני 3 years54:40

54:40

https://go.dok.community/slack https://dok.community ABSTRACT OF THE TALK This talk will provide a high-level overview of Kubernetes, Helm charts and how they can be used to deploy Apache Druid clusters of any size. We'll review how Kubernetes functionality enables resilience and self-healing, historical tiers through node group affinity, middle manager scaling through Kubernetes autoscaling to optimize ingestion capacity and some of the gotchas along the way. BIO Sergio Ferragut is a database veteran turned Developer Advocate at Imply. His experience includes 16 years at Teradata in professional services and engineering roles. He has direct experience in building analytics applications spanning the retail, supply chain, pricing optimization and IoT spaces. Sergio has worked at multiple technology start-ups including APL and Splice Machine where he helped guide product design and field messaging.…

Data on Kubernetes Community

1
Dok Talks #123 - Can Data Become a Declarative Resource? // Roey Libfeld, Michael Greenberg & Uri Zaidenwerg 1:07:10

לפני 3 years1:07:10

1:07:10

https://go.dok.community/slack https://dok.community ABSTRACT OF THE TALK Most K8s users find stateful K8s deployments challenging, to say the least, when persistent data is involved the declarative, portable, self-healing nature that we are accustomed to in stateless deployments is replaced with complexity, resiliency issues and portability restrictions. While K8s is a standard, portable between environments, data is restricted to its physical location and its provider. In this session, we will review the challenges facing stateful K8s, and how to free data to enjoy the same level of portability as the application layer. KEY TAKE-AWAYS FROM THE TALK To assure stateful deployments enjoy the same declarative nature we came to love in stateless deployments, we the K8s users should adopt technologies that do not restrict data to a physical location or to a specific public cloud vendor, designing portable agnostic K8s apps should be…

Data on Kubernetes Community

1
Dok Talks #122 - Operationalizing a Data Infrastructure Stack on Kubernetes 36:35

לפני 3 years36:35

36:35

https://go.dok.community/slack https://dok.community ABSTRACT OF THE TALK Kubernetes is massively powerful, but there are still a large number of details that are needed to get right before really leveraging to build a robust data infrastructure. I'll dive into the complexities we've uncovered at Plural and how we're solving them, from cloud configuration to observability to managing upgrade lifecycles. BIO Career backend engineer turned tech founder…

Data on Kubernetes Community

1
Dok Student Sessions - Contributing to Cloud Native Glossary // Kunal Verma 38:05

לפני 3 years38:05

38:05

https://go.dok.community/slack https://dok.community ABSTRACT OF THE TALK In this session, we'll be talking about a new open source project in the CNCF community i.e. the Cloud Native Glossary. The main aim of the glossary is to be the centre-point for all the terms related to the cloud-native ecosystem & we are actively looking for awesome contributors from the community. I'll be talking about how one can get involved & start their contributions in this initiative. If you are someone wanting to get involved with the CNCF community, this is your chance!!! BIO I'm Kunal Verma, a sophomore pursuing Bachelor's in IT from India. I am a DevOps enthusiast and also passionate about Open-source! Currently, I am exploring the DevOps and cloud-native ecosystem and contributing to the Kubernetes, Data On Kubernetes Community & the Layer5 community. Really like to interact with folks and be a part of awesome communities and learn more and more!…

Data on Kubernetes Community

1
Dok Talks #121 - Running Stateful Apps in Kubernetes Made Simple // Steve Buchanan 1:00:40

לפני 3 years1:00:40

1:00:40

https://go.dok.community/slack https://dok.community ABSTRACT OF THE TALK Eventually the time will come to run a stateful app in Kubernetes. This can be a scary thing adding more moving parts to a Kubernetes cluster and deploying as well as managing your app on Kubernetes when it requires state. In this talk Steve Buchanan will take you through a journey of understanding how storage works in Kubernetes, how to Persistent state with pods, what storage options are available with Azure Kubernetes Service, best practices, and a demo of deploying a stateful app to AKS. BIO Steve Buchanan is a Principal Program Manager with a leading global tech giant focused on improving the cloud. He is a Pluralsight author, the author of eight technical books, and a former 10-time Microsoft MVP. He has presented at tech events, including, DevOps Days, Open Source North, Midwest Management Summit (MMS), Microsoft Ignite, BITCon, Experts Live Europe, OSCON, Inside Azure management, and user groups. He has been featured in several publications including the Star Tribune (the 5th largest newspaper in the US). He stays active in the technical community and enjoys blogging about his adventures in the world of IT at www.buchatech.com KEY TAKE-AWAYS FROM THE TALK Overview of Storage in Kubernetes covering Storage Classes, Persistent Volumes, & Persistent Volume Claims. Overview of Azure Storage, Best Practices to running stateful apps in Kubernetes.…

Data on Kubernetes Community

1
Dok Talks #120 - A Gentle Introduction to Building Data Intensive Applications // Joe Karlsson 1:01:50

לפני 3 years1:01:50

1:01:50

https://go.dok.community/slack https://dok.community ABSTRACT OF THE TALK We all know that data intensive applications have had explosive growth in the past decade. Data now drives significant portions of our lives, from making sense of IoT swarm data to detecting fraudulent transactions at your bank. As software developers, we all have unprecedented opportunities to build data applications that generate value from these massive datasets. Join Bart Farrell and Joe Karlsson for this talk as we discuss gotchas and best practices of building deploying and scaling data intensive apps. KEY TAKE-AWAYS FROM THE TALK Build a data-intensive application from scratch using only SQL. We will discuss gotchas and best practices of building deploying and scaling a key-value store. BIO Joe Karlsson (He/They) is a Database Engineer turned Developer Advocate (and massive data nerd) currently working at SingleStore. He empowers developers to think creatively when building applications with a massive amount of data, through demos, blogs, videos, or whatever else developers need. Joe's career has taken him from building out database best practices and demos for MongoDB, architecting and building one largest eCommerce websites in North America at Best Buy, and teaching at one of the most highly-rated software development boot camps on Earth. Joe is also a TEDx Speaker, film buff, and avid [TikToker]…

Data on Kubernetes Community

1
Dok Talks #119 - Cloud-Native Data Pipelines // Hakan Lofcali 53:25

לפני 3 years53:25

53:25

https://go.dok.community/slack https://dok.community ABSTRACT OF THE TALK This talk walks you through our stack, architecture, and processes. We develop tools to deploy and run data-driven applications in a cloud-native environment. We will give a whirlwind tour on developing a Java Quarkus application, a CICD stack powered by GitHub Actions / ArgoCD, building and deploying containerized Kafka Streams applications at runtime with Jib container builder. Having introduced the above common understanding, we will give a high-level overview of how we utilize modern Kubernetes and Cloud tooling to manage multiple clusters in different organizations together with our customers. BIO DataCater commoditizes data pipeline development lifecycle by applying software engineering and cloud native practices to data work. Hakan is a Software / Data Engineer and CTO of DataCater. He worked and built his knowledge around Software, Data Engineering, and Cloud-Native Computing in severely different environments. From early start-up to hyper-scaler AWS. From sports media companies to highly regulated FSI enterprises. The experiences gained, problems encountered, and solutions found led to him co-founding DataCater to enhance tooling in the Data space.…

Data on Kubernetes Community

1
Dok Talks #118 - Troubleshooting ClickHouse Performance // Shiv Lyer 1:02:50

לפני 3 years1:02:50

1:02:50

https://go.dok.community/slack https://dok.community ABSTRACT OF THE TALK This talk is about how I use several tools, technologies and processes to troubleshoot ClicHouse Performance. I will be talking about multiple Linux Toolkits, Trace Profilers like DTrace, BPF etc. and also ClickHouse System Tables. This talk also covers best practices/checklist / run-book for building "High Performance ClickHouse Infrastructure Operations" BIO Open Source Database Systems Geek in MySQL, MariaDB, PostgreSQL and ClickHouse with core expertise in performance, scalability, high availability and database reliability engineering, Shiv currently is the founder and principal of MinervaDB Inc., an enterprise-class 24*7 Consultative Support and Managed Services Provider for MySQL, MariaDB and PostgreSQL. Shiv also is the Founder and Principal of ChistaDATA Inc., an independent 24*7 Consultative Support and Managed Service Provider for ClickHouse. Shiv in the past worked for companies like MySQL AB, SUN Microsystems, AOL, eBay, PayPal, PalominoDB and Percona. Shiv also is a frequent speaker in open source conferences worldwide. KEY TAKE-AWAYS FROM THE TALK Troubleshooting ClickHouse Performance intuitively…

Data on Kubernetes Community

1
Dok Talks #117 - Why you should care about data mesh // Luke Feeney 58:40

לפני 3 years58:40

58:40

https://go.dok.community/slack https://dok.community ABSTRACT OF THE TALK Data mesh is a new approach for designing modern data architectures by embracing organizational constructs as well as technological aspects, data management, governance, etc. The idea is that data should be easily accessible and interconnected across the entire business. This talk will look at the concept of data mesh architecture. It will discuss implementing data mesh architecture in organizations and will relate data mesh to updates at TerminusDB. BIO Co-founder of TerminusDB, the open source document graph database. KEY TAKE-AWAYS FROM THE TALK Data mesh TerminusDB Document graphs Git for data https://terminusdb.com/blog/…

Data on Kubernetes Community

1
Dok Specials - Ask Us Anything About Postgres // Gabriele Bartolini, Ryan Booz & Álvaro Hernández 1:03:30

לפני 3 years1:03:30

1:03:30

https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK What's the deal with Postgres in Kubernetes? To get some answers as well as more questions, we're bringing together Álvaro Hernandez from Ongres, Ryan Booz from Timescale, and Gabriele Bartolini from EDB for an "Ask Us Anything" session. Send us your questions on Slack!…

Data on Kubernetes Community

1
Dok Specials - Ask Patrick and Jeff Anything About Data on Kubernetes // Patrick McFadin & Jeff Carpenter 1:02:15

לפני 3 years1:02:15

1:02:15

https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK Patrick is a Data on Kubernetes Community veteran. He did the very first session "Is k8s even ready for data?" in July 2020 and has seen the growth of the community since then. Jeff Carpenter is a Software Engineer at Datastax where he works on the S targate.io project. If you have questions you want to be answered in the session, please feel free to message Bart on Slack.…

Data on Kubernetes Community

1
Dok Specials - Unravel the key to your Kubernetes secrets 2:14:47

לפני 3 years2:14:47

2:14:47

https://go.dok.community/slack https://dok.community/

Data on Kubernetes Community

1
Dok Talks #116 - Nebula Graph: Open Source Distributed Graph Database // Wey (Siwei) Gu 1:07:10

לפני 3 years1:07:10

1:07:10

https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK Nebula Graph Demystified Graph on K8s Know-How of Graph Database BIO Open Source believer, builder, singer and Graph Magic advocate(to scale the magic to help wizards) https://siwei.io/en/about/ KEY TAKE-AWAYS FROM THE TALK Why& How graph db would help real life problems Arch and Design the Nebula Graph Database and the Intro. of the Project Implementation of the Nebula K8s operator Graph Database Know How in actions on: Building an intelligent Voice Robot MetaData governance and Lineage Building a corporation shareholding relations system…

Data on Kubernetes Community

1
Dok Special - Show me the money: The business side of DoK // Evan Powell, Brian Schechter & Misha Herscu 57:10

לפני 3 years57:10

57:10

https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK Running stateful workloads on Kubernetes isn't just a technical question. Without keeping the business value it provides in mind, it becomes a moot point. In order to drive these conversations forward, we'll be joined by Melissa Logan (Director at the DoKC), Evan Powell (Adviser/investor who was instrumental in launching the DoKC), Brian Schecther (Partner at Primary Venture Partners). KEY TAKE-AWAYS FROM THE TALK - What are the economic drivers of Data on Kubernetes adoption? - What are the business or economic drivers in the way of Data on Kubernetes adoption? - How will Data on Kubernetes evolve in the coming years given other open source trends?…

Data on Kubernetes Community

1
Dok Talks #115 - What More Can I Learn From My OpenTelemetry Traces? // John Pruitt 1:00:45

לפני 3 years1:00:45

1:00:45

https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK Of the three observability data types supported by OpenTelemetry (metrics, logs, and traces) the latter is the one with most potential. Tracing gives users insights into how requests are processed by microservices in a modern, cloud-native architecture. Jaeger and Grafana can visualize a single trace, showing how an individual request traversed your entire system. This helps for distributed debugging and analysis, but using traces only this way is limiting. What if you stored tracing data in a SQL database? You could ask global questions about your system. You could find slow communication paths, where the error rate spiked since the last deployment, or where the request rate suddenly dropped. Thus, tracing can be used proactively to help you spot issues before your customers do. This talk will show you how to do all the above by ingesting OpenTelemetry traces into a PostgreSQL/TimescaleDB database, and building custom dashboards using SQL to make the most out of your tracing data. BIO John Pruitt is a software engineer at Timescale. His work focuses on database/SQL development for the Promscale open-source observability tool, and currently on adding support for OpenTelemetry tracing. Prior to joining Timescale, John grew the DBA team at Shipt. Most of the balance of his career was spent building custom time-series applications in the energy industry and leading data warehousing efforts at regional banks. KEY TAKE-AWAYS FROM THE TALK - What is distributed tracing - Why viewing individual traces is of limited value - How SQL can be used to analyze and visualize traces - What insights can be unlocked using SQL against traces…

Data on Kubernetes Community

1
Dok Talks #114 - Helm for Beginners with Portainer // Hrittik Roy 51:55

לפני 3 years51:55

51:55

https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK Helm allows you to install packages in your cluster, much like you would use apt, yum on your laptop. Just define the components that you want to install in your application, and it will take care of the installation and configuration of these components, saving you from headaches. This talk will go through the package manager for Kubernetes and its components. In the end, we would deploy an application with Helm to our cluster with Portainer. BIO Hrittik is a Portainer Advocate, a software engineer at Ayedo GmbH and a student in his sophomore year. Hrittik focuses on building tech communities and software with the help of cloud native technologies. He has been a speaker at conferences like the Cloud Summit and conducted various events that have collectively empowered students and professionals on DevOps and Cloud. Connect with him here: https://linktr.ee/hrittikhere KEY TAKE-AWAYS FROM THE TALK - Understand Helm, Charts and Repositories - Understand the need for the package manager - Install Application with Helm via Portainer…

Data on Kubernetes Community

1
Dok Talks #113 - Developing Stateful Application on Kubernetes // Rob Pacheco 53:15

לפני 3 years53:15

53:15

https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK Modern web applications are typically comprised of multiple services which utilize storage in a variety of ways. Utilizing storage in Kubernetes introduces challenges that are not obvious while developing locally. We’ll introduce these challenges and show good and bad ways to handle them. We’ll dive into the details of an application along with its deployment in Kubernetes to understand why certain storage patterns are problematic and how they can be modified to behave well in a Kubernetes-based deployment. BIO Rob Pacheco is currently leading cloud operations at Rumble. Prior to Rumble, Rob spent time at Vision Government Solutions as well as Black Duck Software re-architecting, securing, and containerizing its products to run within containerized environments, including Kubernetes and Google Kubernetes Engine. Rob is also the author of the liveProject “Creating and Managing Cloud Native Services in Kubernetes”, and the upcoming liveVideo “Surviving Kubernetes Deployments as an Application Developer” KEY TAKE-AWAYS FROM THE TALK * Behavior stateful primitives in Kubernetes * Common storage patterns in web applications * Patterns for well-behaved storage in a Kubernetes cluster…

Data on Kubernetes Community

1
Dok Talks #112 - Production Postgres Made Easy on Kubernetes // Jonathan Katz 1:02:45

לפני 3 years1:02:45

1:02:45

https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK It's one thing to get a database up and running on Kubernetes; it's another to run it in production. There are many "Day 2" features to think about when running any database in production, including high availability, backups + RTO/RPO, monitoring, security, and disaster recovery. Combining Kubernetes with the Operator pattern can help make this easier, and add advanced features like self-healing, simple scaling, secure credential management, and more. PGO, the open source Postgres Operator from Crunchy Data, makes it easy to manage production Postgres clusters on Kubernetes the GitOps way. Using simple manifests, PGO deploys HA Postgres clusters with self-healing capabilities that apps can securely connect to. In this talk we will review the important concepts to know for running databases in production, how PGO implements these features, and then spend most of the time demoing how all of the key "Day 2" features work with a PGO-managed Postgres databases connected with an application! BIO Jonathan S. Katz is the VP of Platform Engineering at Crunchy Data, with a focus on managing PGO, an open source Postgres Operator. Jonathan is a member of the PostgreSQL Core Team and involved in various governance aspects of the PostgreSQL Global Development Group. He serves as a Secretary and Director of the nonprofit PostgreSQL Community Association of Canada and is a Director of the nonprofit United States PostgreSQL Association. He has given talks at conferences all over the world for over a decade on topics involving Postgres, from application developer uses to operations, particularly focusing on Kubernetes over the past several years. Prior to Crunchy Data, Jonathan was CTO at VenueBook, and before that, VP of Technology at Paperless Post. At both companies Jonathan developed robust platforms using PostgreSQL, taking advantage of its many features, from complex data types to its ability to stream logical changes. Jonathan graduated from Tufts University with a B.S. in Computer Science and a B.A. in Mathematics.…

Data on Kubernetes Community

1
Dok Talks #111 - Scheduled Scaling with Dask and Argo Workflows 1:05:30

לפני 3 years1:05:30

1:05:30

https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK Complex computational workloads in Python are a common sight these days, especially in the context of processing large and complex datasets. Battle-hardened modules such as Numpy, Pandas, and Scikit-Learn can perform low-level tasks, while tools like Dask makes it easy to parallelize these workloads across distributed computational environments. Meanwhile, Argo Workflows offers a Kubernetes-native solution to provisioning cloud resources in Kubernetes and triggering workflows on a regular schedule. Being Kubernetes-native, Argo Workflows also meshes nicely with other Kubernetes tools. This talk discusses the combination of these two worlds by showcasing a set-up for Argo-managed workflows which schedule and automatically scale-out Dask-powered data pipelines in Python. BIO Former academic in the field of renewable energy simulation and energy systems analysis. Currently responsible for architecting and maintaining the cloud- and data strategy at ACCURE Battery Intelligence KEY TAKE-AWAYS FROM THE TALK Argo Workflows + Dask is a nice combination for data-processing pipelines. There are a a few "gotchyas" to be on the look-out for, but in nevertheless this is still a generally-applicable and powerful combination. https://github.com/sevberg…

Data on Kubernetes Community

1
Dok Talks #110 - Overview of the Oracle Database Operator for Kubernetes // Paulo Simoes 44:25

לפני 3 years44:25

44:25

https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK An introduction to OraOperator, the open source project that makes Oracle Database Kubernetes-Native. BIO Paulo works as the Developer Evangelist at Oracle focuses on Microservices, Application Development with Converged Oracle Database, and Event Mesh. His current technical expertise focuses on Cloud Native architecture and best practices, including containerization and microservices development. KEY TAKE-AWAYS FROM THE TALK Oracle Database as a Kubernetes-native platform. Simplified way to operator Oracle Database.…

Data on Kubernetes Community

1
Dok Talks #109 - Benchmarking for PostgreSQL workloads in Kubernetes / Part 2 // Gabriele Bartolini 1:04:35

לפני 4 years1:04:35

1:04:35

https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK 6 months have passed since our last DoK webinar about benchmarking PostgreSQL workloads in a Kubernetes environment. In the meantime, many things have happened at EDB, and we’re happy to share what we’ve learned in this timeframe. We’ll use cnp-bench and cnp-sandbox to help us describe some of the challenges we might face when running PostgreSQL workloads, how to spot them, and what actions to take to make your databases healthier and more longeve. cnp-bench is a collection of Helm charts that help run storage and database benchmarks, using popular open source tools like fio, pgbench, and HammerDB. cnp-sandbox is a Helm chart that sets up a Prometheus/Grafana stack, including basic metrics and dashboards for Cloud Native PostgreSQL, the Kubernetes operator developed by EDB. Both cnp-sandbox and cnp-bench are open source and recommended for development, testing, and pre-production environments only. BIO A long time open-source programmer and entrepreneur, Gabriele has a degree in Statistics from the University of Florence. After having consistently contributed to the growth of 2ndQuadrant and its members through nurturing a lean and devops culture, he is now leading the Cloud Native initiative at EDB. Gabriele lives in Prato, a small but vibrant city located in the northern part of Tuscany, Italy - famous for having hosted the first European PostgreSQL conferences. His second home is Melbourne, Australia, where he studied at Monash University and worked in the ICT sector. He loves playing the Blues with his Fender Stratocaster, but his major passions are called Elisabeth and Charlotte! KEY TAKE-AWAYS FROM THE TALK - A methodology for benchmarking a PostgreSQL database in Kubernetes - Open source set of tools for benchmarking a PostgreSQL database in Kubernetes - Reasons why benchmarking both the storage and the database is important https://github.com/EnterpriseDB/cnp-sandbox https://github.com/EnterpriseDB/cnp-bench…

Data on Kubernetes Community

1
Dok Talks #108 - Postgres on Kubernetes Applied at Scale in Zalando // Álvaro Hernández & Alexander Kukushkin 1:02:20

לפני 4 years1:02:20

1:02:20

https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK Some of the topics that may or may not be discussed on this live talk could be: - Other than Patroni and the operator, what else was needed to fully switch to Kubernetes? - What is left, what is the future going to bring to the world of Kubernetes and Postgres operators? Expect strong opinions, and facts. Live. Uncensored. Postgres. On. Kubernetes. BIO Álvaro is a passionate database and software developer. He founded and works as the Founder & CEO of OnGres ( https://ongres.com ). He has been dedicated to PostgreSQL and R&D in databases for two decades. An open source advocate and developer at heart, Álvaro is a well-known member of the PostgreSQL Community, to which he has contributed founding the non-profit Fundación PostgreSQL ( https://postgresql.fund ) and the Spanish PostgreSQL User Group. You can find him frequently speaking at PostgreSQL, database, cloud, and Java conferences. Every year, Álvaro travels approximately three-four times around the globe—in 2020 (or now more likely 2021 due to COVID-19), he will hit the milestone of having delivered 100 tech talks. AWS Data Hero (2019). ------------ During his professional career, Alexander touched PostgreSQL from all possible sides: as a Web Developer, as System Administrator, and as a Database Engineer now. Five years ago he started working on Patroni project and achieved quite a big success with it. Nowadays Patroni is used in the heart of PostgreSQL-as-a-Service on Kubernetes at Zalando to run 1000+ PostgreSQL HA clusters. Besides that, Alexander periodically reports bugs and contributes patches to PostgreSQL and some other open-source projects (usually Postgres related).…

Data on Kubernetes Community

1
DoK Talks #107- Cross region replication in Percona Kubernetes Operators // Ege Gunes 45:40

לפני 4 years45:40

45:40

https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK With Percona Kubernetes Operators you can migrate your data to Kubernetes or have a disaster recovery site to recover it in unfortunate times. I'll talk about the implementation, challenges and caveats of such setups and what the future may bring.…

Data on Kubernetes Community

1
DoK Talks #106- Cloud native data warehousing with Kubernetes // Mark Cusack & Matthew Ripley 59:25

לפני 4 years59:25

59:25

https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK Yellowbrick's cloud data warehouse is natively integrated with Kubernetes. Not only does Kubernetes provide the elasticity, resilience and upgradeability expected of a modern data warehouse, it also enables Yellowbrick to be deployed anywhere: in public clouds, private clouds and at the network edge. During this talk, we will dive into the details of how we use Kubernetes to power our data warehouse solution, how we've created a SQL interface to Kubernetes, and give a live demo. We'll also talk about the business value that enterprises can derive from such a flexible solution built on Kubernetes. KEY TAKE-AWAYS FROM THE TALK You'll come away with: an understanding of the business value derived from a Kubernetes-based data warehouse; insight into our SQL interface to Kubernetes; and the overall capabilities of Yellowbrick BIO Before joining Yellowbrick as CTO, Mark was Vice President for Data and Analytics at Teradata, where he led a variety of product management and technology teams in data warehouse and advanced analytics groups. He was also Chief Architect of Teradata’s IoT Analytics effort. Mark joined Teradata in 2014 when Teradata acquired the startup RainStor, where he was a co-founding developer and Chief Architect. Prior to RainStor, Mark was a lead scientist in the UK Ministry of Defence. Mark holds a Ph.D. in computational physics from Newcastle University, UK, with a thesis centered on discovering the electronic and non-linear optical properties of quantum dots. As a research fellow at Newcastle, he developed new techniques to model these novel quantum structures using large-scale parallel and distributed computing approaches..…

Data on Kubernetes Community

1
DoK Special: Mental Health and Covid-19: Retrospective and Perspective // Andrea Dobson, Erin Grinshteyn and Julia simon 59:15

לפני 4 years59:15

59:15

https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK Fear, anxiety, uncertainty, frustration- any of these sound familiar since the pandemic started? How is your organization addressing the challenges of Covid-19 on mental health? We will start this session with an overview from Erin Grinshteyn, who is an Associate Professor at the University of San Francisco and has done research about the impact of fear in various collectives. From there we will have Andrea Dobson, Head of People at Container Solutions who is also a psychologist, and Julia Simon, HR Professional and Leader at the #burnout group in the CNCF…

ברוכים הבאים אל Player FM!

Player FM סורק את האינטרנט עבור פודקאסטים באיכות גבוהה בשבילכם כדי שתהנו מהם כרגע. זה יישום הפודקאסט הטוב ביותר והוא עובד על אנדרואיד, iPhone ואינטרנט. הירשמו לסנכרון מנויים במכשירים שונים.