32 subscribers
התחל במצב לא מקוון עם האפליקציה Player FM !
פודקאסטים ששווה להאזין
בחסות


Rethinking Apache Kafka Security and Account Management
Manage episode 424666722 series 2510642
Is there a better way to manage access to resources without compromising security? New employees need access to a variety of resources within a company's tech stack. But manually granting access can be error-prone. And when employees leave, their access must be revoked, thus potentially introducing security risks if an admin misses one. In this podcast, Kris Jenkins talks to Anuj Sawani (Security Product Manager, Confluent) about the centralized identity management system he helped build to integrate with Apache Kafka® to prevent common identity management headaches and security risks.
With 12+ years of experience building cybersecurity products for enterprise companies, Anuj Sawani explains how he helped build out KIP-768 (Secured OAuth support in Kafka) that supports a unified identity mechanism that spans across cloud and on-premises (hybrid scenarios).
Confluent Cloud customers wanted a single identity to access all their services. The manual process required managing different sets of identity stores across the ecosystem. Anuj goes on to explain how Identity and Access Management (IAM) using cloud-native authentication protocols, such as OAuth or OpenID Connect, solves this problem by centralizing identity and minimizing security risks.
Anuj emphasizes that sticking with industry standards is key because it makes integrating with other systems easy. With OAuth now supported in Kafka, this means performing client upgrades, configuring identity providers, etc. to ensure the applications can leverage new capabilities. Some examples of how to do this are to use centralized identities for client/broker connections.
As Anuj continues to build and enhance features, he hopes to recommend this unified solution to other technology vendors because it makes integration much easier. The goal is to create a web of connectors that support the same standards. The future is bright, as other organizations are researching supporting OAuth and similar industry standards. Anuj is looking forward to the evolution and applying it to other use cases and scenarios.
EPISODE LINKS
- Introduction to Confluent Cloud Security
- KIP-768: Secured OAuth support in Apache Kafka
- Confluent Cloud Documentation: OAuth 2.0 Support
- Apache Kafka Security Best Practices
- Security for Real-Time Data Stream Processing with Confluent Cloud
- Watch the video version of this podcast
- Kris Jenkins’ Twitter
- Streaming Audio Playlist
- Join the Confluent Community
- Learn more with Kafka tutorials, resources, and guides at Confluent Developer
- Live demo: Intro to Event-Driven Microservices with Confluent
- Use PODCAST100 to get an additional $100 of free Confluent Cloud usage (details)
פרקים
1. Intro (00:00:00)
2. Common identity management problems and security risks (00:06:19)
3. Building a centralized identity management system (00:11:33)
4. Recommendations for enterprise IAM (00:14:47)
5. OAuth vs. Open ID Connect (00:18:35)
6. Integrating identity providers with Apache Kafka (00:22:36)
7. KIP-768: Introducing secured OAuth support for Apache Kafka (00:25:24)
8. Setting up discovery end points (00:30:35)
9. Tips for getting started with centralized identity management (00:35:22)
10. Authentication vs. authorization standards (00:38:41)
11. It's a wrap! (00:39:42)
265 פרקים
Manage episode 424666722 series 2510642
Is there a better way to manage access to resources without compromising security? New employees need access to a variety of resources within a company's tech stack. But manually granting access can be error-prone. And when employees leave, their access must be revoked, thus potentially introducing security risks if an admin misses one. In this podcast, Kris Jenkins talks to Anuj Sawani (Security Product Manager, Confluent) about the centralized identity management system he helped build to integrate with Apache Kafka® to prevent common identity management headaches and security risks.
With 12+ years of experience building cybersecurity products for enterprise companies, Anuj Sawani explains how he helped build out KIP-768 (Secured OAuth support in Kafka) that supports a unified identity mechanism that spans across cloud and on-premises (hybrid scenarios).
Confluent Cloud customers wanted a single identity to access all their services. The manual process required managing different sets of identity stores across the ecosystem. Anuj goes on to explain how Identity and Access Management (IAM) using cloud-native authentication protocols, such as OAuth or OpenID Connect, solves this problem by centralizing identity and minimizing security risks.
Anuj emphasizes that sticking with industry standards is key because it makes integrating with other systems easy. With OAuth now supported in Kafka, this means performing client upgrades, configuring identity providers, etc. to ensure the applications can leverage new capabilities. Some examples of how to do this are to use centralized identities for client/broker connections.
As Anuj continues to build and enhance features, he hopes to recommend this unified solution to other technology vendors because it makes integration much easier. The goal is to create a web of connectors that support the same standards. The future is bright, as other organizations are researching supporting OAuth and similar industry standards. Anuj is looking forward to the evolution and applying it to other use cases and scenarios.
EPISODE LINKS
- Introduction to Confluent Cloud Security
- KIP-768: Secured OAuth support in Apache Kafka
- Confluent Cloud Documentation: OAuth 2.0 Support
- Apache Kafka Security Best Practices
- Security for Real-Time Data Stream Processing with Confluent Cloud
- Watch the video version of this podcast
- Kris Jenkins’ Twitter
- Streaming Audio Playlist
- Join the Confluent Community
- Learn more with Kafka tutorials, resources, and guides at Confluent Developer
- Live demo: Intro to Event-Driven Microservices with Confluent
- Use PODCAST100 to get an additional $100 of free Confluent Cloud usage (details)
פרקים
1. Intro (00:00:00)
2. Common identity management problems and security risks (00:06:19)
3. Building a centralized identity management system (00:11:33)
4. Recommendations for enterprise IAM (00:14:47)
5. OAuth vs. Open ID Connect (00:18:35)
6. Integrating identity providers with Apache Kafka (00:22:36)
7. KIP-768: Introducing secured OAuth support for Apache Kafka (00:25:24)
8. Setting up discovery end points (00:30:35)
9. Tips for getting started with centralized identity management (00:35:22)
10. Authentication vs. authorization standards (00:38:41)
11. It's a wrap! (00:39:42)
265 פרקים
כל הפרקים
×
1 Apache Kafka 3.5 - Kafka Core, Connect, Streams, & Client Updates 11:25

1 How to use Data Contracts for Long-Term Schema Management 57:28

1 How to use Python with Apache Kafka 31:57

1 Next-Gen Data Modeling, Integrity, and Governance with YODA 55:55

1 Migrate Your Kafka Cluster with Minimal Downtime 1:01:30

1 Real-Time Data Transformation and Analytics with dbt Labs 43:41

1 What is the Future of Streaming Data? 41:29

1 What can Apache Kafka Developers learn from Online Gaming? 55:32

1 How to use OpenTelemetry to Trace and Monitor Apache Kafka Systems 50:01

1 What is Data Democratization and Why is it Important? 47:27

1 Git for Data: Managing Data like Code with lakeFS 30:42

1 Using Kafka-Leader-Election to Improve Scalability and Performance 51:06

1 Real-Time Machine Learning and Smarter AI with Data Streaming 38:56

1 The Present and Future of Stream Processing 31:19

1 Top 6 Worst Apache Kafka JIRA Bugs 1:10:58

1 Learn How Stream-Processing Works The Simplest Way Possible 31:29

1 Building and Designing Events and Event Streams with Apache Kafka 53:06

1 Rethinking Apache Kafka Security and Account Management 41:23

1 Real-time Threat Detection Using Machine Learning and Apache Kafka 29:18

1 Improving Apache Kafka Scalability and Elasticity with Tiered Storage 29:32

1 Decoupling with Event-Driven Architecture 38:38

1 If Streaming Is the Answer, Why Are We Still Doing Batch? 43:58

1 Security for Real-Time Data Stream Processing with Confluent Cloud 48:33

1 Running Apache Kafka in Production 58:44

1 Build a Real Time AI Data Platform with Apache Kafka 37:18

1 Optimizing Apache JVMs for Apache Kafka 1:11:42


1 Application Data Streaming with Apache Kafka and Swim 39:10

1 International Podcast Day - Apache Kafka Edition | Streaming Audio Special 1:02:22


1 Real-Time Stream Processing, Monitoring, and Analytics With Apache Kafka 34:07

1 Reddit Sentiment Analysis with Apache Kafka-Based Microservices 35:23

1 Capacity Planning Your Apache Kafka Cluster 1:01:54

1 Streaming Real-Time Sporting Analytics for World Table Tennis 34:29

1 Real-Time Event Distribution with Data Mesh 48:59

1 Apache Kafka Security Best Practices 39:10

1 What Could Go Wrong with a Kafka JDBC Connector? 41:10

1 Apache Kafka Networking with Confluent Cloud 37:22

1 Event-Driven Systems and Agile Operations 53:22

1 Streaming Analytics and Real-Time Signal Processing with Apache Kafka 1:06:33

1 Blockchain Data Integration with Apache Kafka 50:59

1 Automating Multi-Cloud Apache Kafka Cluster Rollouts 48:29

1 Common Apache Kafka Mistakes to Avoid 1:09:43

1 Tips For Writing Abstracts and Speaking at Conferences 48:56

1 How I Became a Developer Advocate 29:48

1 Data Mesh Architecture: A Modern Distributed Data Model 48:42

1 Flink vs Kafka Streams/ksqlDB: Comparing Stream Processing Tools 55:55

1 Practical Data Pipeline: Build a Plant Monitoring System with ksqlDB 33:56


1 Scaling Apache Kafka Clusters on Confluent Cloud ft. Ajit Yagaty and Aashish Kohli 49:07

1 Streaming Analytics on 50M Events Per Day with Confluent Cloud at Picnic 34:41


1 Optimizing Apache Kafka's Internals with Its Co-Creator Jun Rao 48:54

1 Using Event-Driven Design with Apache Kafka Streaming Applications ft. Bobby Calderwood 51:09

1 Monitoring Extreme-Scale Apache Kafka Using eBPF at New Relic 38:25

1 Confluent Platform 7.1: New Features + Updates 10:01

1 Scaling an Apache Kafka Based Architecture at Therapie Clinic 1:10:56

1 Bridging Frontend and Backend with GraphQL and Apache Kafka ft. Gerard Klijs 23:13

1 Building Real-Time Data Governance at Scale with Apache Kafka ft. Tushar Thole 42:58

1 Handling 2 Million Apache Kafka Messages Per Second at Honeycomb 41:36


1 Serverless Stream Processing with Apache Kafka ft. Bill Bejeck 42:23

1 The Evolution of Apache Kafka: From In-House Infrastructure to Managed Cloud Service ft. Jay Kreps 46:32


1 Intro to Event Sourcing with Apache Kafka ft. Anna McDonald 30:14

1 Expanding Apache Kafka Multi-Tenancy for Cloud-Native Systems ft. Anna Povzner and Anastasia Vela 31:01


1 Optimizing Cloud-Native Apache Kafka Performance ft. Alok Nikhil and Adithya Chandra 30:40

1 From Batch to Real-Time: Tips for Streaming Data Pipelines with Apache Kafka ft. Danica Fine 29:50

1 Real-Time Change Data Capture and Data Integration with Apache Kafka and Qlik 34:51

1 Modernizing Banking Architectures with Apache Kafka ft. Fotios Filacouris 34:59

1 Running Hundreds of Stream Processing Applications with Apache Kafka at Wise 31:08

1 Lessons Learned From Designing Serverless Apache Kafka ft. Prachetaa Raghavan 28:20

1 Using Apache Kafka as Cloud-Native Data System ft. Gwen Shapira 33:57

1 ksqlDB Fundamentals: How Apache Kafka, SQL, and ksqlDB Work Together ft. Simon Aubury 30:42

1 Explaining Stream Processing and Apache Kafka ft. Eugene Meidinger 29:28

1 Handling Message Errors and Dead Letter Queues in Apache Kafka ft. Jason Bell 37:41

1 Confluent Platform 7.0: New Features + Updates 12:16

1 Real-Time Stream Processing with Kafka Streams ft. Bill Bejeck 35:32

1 Automating Infrastructure as Code with Apache Kafka and Confluent ft. Rosemary Wang 30:08

1 Getting Started with Spring for Apache Kafka ft. Viktor Gamov 32:44

1 Powering Event-Driven Architectures on Microsoft Azure with Confluent 38:42

1 Automating DevOps for Apache Kafka and Confluent ft. Pere Urbón-Bayes 26:08

1 Intro to Kafka Connect: Core Components and Architecture ft. Robin Moffatt 31:18

1 Designing a Cluster Rollout Management System for Apache Kafka ft. Twesha Modi 30:08

1 Apache Kafka 3.0 - Improving KRaft and an Overview of New Features 15:17

1 How to Build a Strong Developer Community with Global Engagement ft. Robin Moffatt and Ale Murray 35:18

1 What Is Data Mesh, and How Does it Work? ft. Zhamak Dehghani 34:56

1 Multi-Cluster Apache Kafka with Cluster Linking ft. Nikhil Bhatia 31:04

1 Using Apache Kafka and ksqlDB for Data Replication at Bolt 29:15

1 Placing Apache Kafka at the Heart of a Data Revolution at Saxo Bank 28:37

1 Advanced Stream Processing with ksqlDB ft. Michael Drogalis 28:26

1 Minimizing Software Speciation with ksqlDB and Kafka Streams ft. Mitch Seymour 31:32

1 Collecting Data with a Custom SIEM System Built on Apache Kafka and Kafka Connect ft. Vitalii Rudenskyi 25:14

1 Consistent, Complete Distributed Stream Processing ft. Guozhang Wang 29:00

1 Powering Real-Time Analytics with Apache Kafka and Rockset 25:44

1 Automated Event-Driven Architectures and Microservices with Apache Kafka and SmartBear 29:53

1 Data-Driven Digitalization with Apache Kafka in the Food Industry at BAADER 27:53

1 Chaos Engineering with Apache Kafka and Gremlin 35:32

1 Boosting Security for Apache Kafka with Confluent Cloud Private Link ft. Dan LaMotte 25:55


1 Adopting OpenTelemetry in Confluent and Beyond ft. Xavier Léauté 32:52

1 Becoming Data Driven with Apache Kafka and Stream Processing ft. Daniel Jagielski 48:10

1 Integrating Spring Boot with Apache Kafka ft. Viktor Gamov 45:08


1 Building a Microservices Architecture with Apache Kafka at Nationwide Building Society ft. Rob Jackson 48:54

1 Examining Apache Kafka Performance Metrics ft. Alok Nikhil 50:30

1 Distributed Systems Engineering with Apache Kafka ft. Guozhang Wang 44:52

1 Scaling Developer Productivity with Apache Kafka ft. Mohinish Shaikh 34:19

1 Change Data Capture and Kafka Connect on Microsoft Azure ft. Abhishek Gupta 43:04

1 Event Streaming Trends and Predictions for 2021 ft. Gwen Shapira, Ben Stopford, and Michael Noll 44:34

1 How to Become a Certified Apache Kafka Expert ft. Niamh O’Byrne and Barry Ballard 43:36

1 Mastering DevOps with Apache Kafka, Kubernetes, and Confluent Cloud ft. Rick Spurgeon and Allison Walther 46:18

1 Apache Kafka 2.7 - Overview of Latest Features, Updates, and KIPs 10:59

1 Choreographing the Saga Pattern in Microservices ft. Chris Richardson 47:49

1 Apache Kafka and Porsche: Fast Cars and Fast Data ft. Sridhar Mamella 42:59

1 Tales from the Frontline of Apache Kafka DevOps ft. Jason Bell 1:00:25

1 Multi-Tenancy in Apache Kafka ft. Anna Pozvner 44:19

1 Distributed Systems Engineering with Apache Kafka ft. Roger Hoover 50:24

1 Why Kafka Streams Does Not Use Watermarks ft. Matthias J. Sax 52:20

1 Distributed Systems Engineering with Apache Kafka ft. Apurva Mehta 49:15

1 Most Terrifying Apache Kafka JIRAs of 2020 ft. Anna McDonald 51:59

1 Ask Confluent #18: The Toughest Questions ft. Anna McDonald 33:46

1 Joining Forces with Spring Boot, Apache Kafka, and Kotlin ft. Josh Long 50:41

1 Building an Apache Kafka Center of Excellence Within Your Organization ft. Neil Buesing 46:22

1 Creating Your Own Kafka Improvement Proposal (KIP) as a Confluent Intern ft. Leah Thomas 46:15

1 Confluent Platform 6.0 | What's New in This Release + Updates 14:11

1 Using Event Modeling to Architect Event-Driven Information Systems ft. Bobby Calderwood 56:41

1 Using Apache Kafka as the Event-Driven System for 1,500 Microservices at Wix ft. Natan Silnitsky 49:12

1 Top 6 Things to Know About Apache Kafka ft. Gwen Shapira 47:27

1 5 Years of Event Streaming and Counting ft. Gwen Shapira, Ben Stopford, and Michael Noll 48:18

1 Championing Serverless Eventing at Google Cloud ft. Jay Smith 47:26

1 Disaster Recovery with Multi-Region Clusters in Confluent Platform ft. Anna McDonald and Mitch Henderson 43:04

1 Developer Advocacy (and Kafka Summit) in the Pandemic Era 41:44

1 Apache Kafka 2.6 - Overview of Latest Features, Updates, and KIPs 10:37

1 Testing ksqlDB Applications ft. Viktor Gamov 39:36

1 How to Measure the Business Value of Confluent Cloud ft. Lyndon Hedderly 54:29

1 Modernizing Inventory Management Technology ft. Sina Sojoodi and Rohit Kelapure 41:32

1 Fault Tolerance and High Availability in Kafka Streams and ksqlDB ft. Matthias J. Sax 54:03

1 Benchmarking Apache Kafka Latency at the 99th Percentile ft. Anna Povzner 46:30

1 Open Source Workflow Automation with Apache Kafka ft. Bernd Ruecker 43:03

1 Growing the Event Streaming Community During COVID-19 ft. Ale Murray 40:19

1 From Monolith to Microservices with Sam Newman 40:27

1 Exploring Event Streaming Use Cases with µKanren ft. Tim Baldridge 51:00

1 Introducing JSON and Protobuf Support ft. David Araujo and Tushar Thole 40:00

1 Scaling Apache Kafka in Retail with Microservices ft. Matt Simpson from Boden 42:01

1 Connecting Snowflake and Apache Kafka ft. Isaac Kunen 31:46

1 AMA with Tim Berglund | Streaming Audio Special 47:09

1 Kubernetes Meets Apache Kafka ft. Kelsey Hightower 42:02

1 Apache Kafka Fundamentals: The Concept of Streams and Tables ft. Michael Noll 48:52

1 IoT Integration and Real-Time Data Correlation with Kafka Connect and Kafka Streams ft. Kai Waehner 40:55

1 Confluent Platform 5.5 | What's New in This Release + Updates 11:20

1 Making Abstract Algebra Count in the World of Event Streaming ft. Sam Ritchie 46:21

1 Apache Kafka 2.5 – Overview of Latest Features, Updates, and KIPs 10:28

1 Streaming Data Integration – Where Development Meets Deployment ft. James Urquhart 55:02

1 How to Run Kafka Streams on Kubernetes ft. Viktor Gamov 41:49

1 Cloud Marketplace Considerations with Dan Rosanova 33:31

1 Explore, Expand, and Extract with 3X Thinking ft. Kent Beck 54:45

1 Ask Confluent #17: The “What is Apache Kafka?” Episode ft. Tim Berglund 25:35

1 Domain-Driven Design and Apache Kafka with Paul Rayner 50:42

1 Machine Learning with TensorFlow and Apache Kafka ft. Chris Mattmann 53:06

1 Distributed Systems Engineering with Apache Kafka ft. Gwen Shapira 48:26

1 Running Apache Kafka Efficiently on the Cloud ft. Adithya Chandra 38:35

1 Engaging Database Partials with Apache Kafka for Distributed System Consistency ft. Pat Helland 42:09

1 The Truth About ZooKeeper Removal and the KIP-500 Release in Apache Kafka ft. Jason Gustafson and Colin McCabe 31:50

1 Resilient Edge Infrastructure for IoT Using Apache Kafka ft. Kai Waehner 27:19

1 Data Management and Digital Transformation with Apache Kafka at Van Oord 28:28

1 Powering Microservices Using Apache Kafka on Node.js with KafkaJS at Klarna ft. Tommy Brunn 31:03

1 Apache Kafka 2.8 - ZooKeeper Removal Update (KIP-500) and Overview of Latest Features 10:48

1 Connecting Azure Cosmos DB with Apache Kafka - Better Together ft. Ryan CrawCour 31:59

1 Automated Cluster Operations in the Cloud ft. Rashmi Prabhu 24:41

1 Resurrecting In-Sync Replicas with Automatic Observer Promotion ft. Anna McDonald 24:33

1 Building Real-Time Data Pipelines with Microsoft Azure, Databricks, and Confluent 30:32

1 Smooth Scaling and Uninterrupted Processing with Apache Kafka ft. Sophie Blee-Goldman 50:33

1 Event-Driven Architecture - Common Mistakes and Valuable Lessons ft. Simon Aubury 42:32

1 The Human Side of Apache Kafka and Microservices ft. SPOUD 45:11

1 Gamified Fitness at Synthesis Software Technologies Using Apache Kafka and IoT 33:32
ברוכים הבאים אל Player FM!
Player FM סורק את האינטרנט עבור פודקאסטים באיכות גבוהה בשבילכם כדי שתהנו מהם כרגע. זה יישום הפודקאסט הטוב ביותר והוא עובד על אנדרואיד, iPhone ואינטרנט. הירשמו לסנכרון מנויים במכשירים שונים.