21,312 subscribers
התחל במצב לא מקוון עם האפליקציה Player FM !
פודקאסטים ששווה להאזין
בחסות


1 The Road Back to Country With Blake Shelton 25:13
Alternatives to Administering and Running Apache Kafka
סדרה בארכיון ("עדכון לא פעיל" status)
When? This feed was archived on March 24, 2016 19:45 (
Why? עדכון לא פעיל status. השרתים שלנו לא הצליחו לאחזר פודקאסט חוקי לזמן ממושך.
What now? You might be able to find a more up-to-date version using the search function. This series will no longer be checked for updates. If you believe this to be in error, please check if the publisher's feed link below is valid and contact support to request the feed be restored or if you have any other concerns about this.
Manage episode 425209805 series 65608


In the past couple of episodes, we’d gone over what Apache Kafka is and along the way we mentioned some of the pains of managing and running Kafka clusters on your own. In this episode, we discuss some of the ways you can offload those responsibilities and focus on writing streaming applications. Along the way, Joe does a mighty fine fill-in for proper noun pronunciation and Allen does a southern auctioneer-style speed talk.
View the full show notes here:
https://www.codingblocks.net/episode237
Reviews
As always, thank you for leaving us a review – we really do appreciate them!
From iTunes: Abucr7
Upcoming Events
Atlanta Dev Con
September 7th, 2024
https://www.atldevcon.com/
DevFest Central Florida on September 28th, 2024
Interested? Submit your talk proposal here:
https://sessionize.com/devfest-florida-orlando-2024/
Kafka Compatible and Kafka Functional Alternatives
Why? Because running any type of infrastructure requires time, knowledge, and blood, sweat and tears
Confluent
- https://www.confluent.io/confluent-cloud/pricing/
- We’ve personally had good experiences with their Kafka as a service
WarpStream
- https://www.warpstream.com/
- “WarpStream is an Apache Kafka® compatible data streaming platform built directly on top of object storage: no inter-AZ bandwidth costs, no disks to manage, and infinitely scalable, all within your VPC”
- ZERO disks to manage
- 10x cheaper than running Kafka
- Agents stream data directly to and from object storage with no buffering on local disks and no data tiering.
- Create new serverless “Virtual Clusters” in our control plane instantly
- Support different environments, teams, or projects without managing any dedicated infrastructure
- Things you won’t have to do with WarpStream
- Upscale a cluster that is about to run out of space
- Figure out how to restore quorum in a Zookeeper cluster or Raft consensus group
- Rebalance partitions in a cluster
- “WarpStream is protocol compatible with Apache Kafka®, so you can keep using all your favorite tools and software. No need to rewrite your application or use a proprietary SDK. Just change the URL in your favorite Kafka client library and start streaming!”
- Never again have to choose between reliability and your budget. WarpStream costs the same regardless of whether you run your workloads in a single availability zone, or distributed across multiple
- WarpStream’s unique cloud native architecture was designed from the ground up around the cheapest and most durable storage available in the cloud: commodity object storage
- WarpStream agents use object storage as the storage layer and the network layer, side-stepping interzone bandwidth costs entirely
- Can be run in BYOC (bring your own cloud) or in Serverless
- BYOC – you provide all the compute and storage – the only thing that WarpStream provides is the control plane
- Data never leaves your environment
- Serverless – fully managed by WarpStream in AWS – will automatically scale for you even down to nothing!
- BYOC – you provide all the compute and storage – the only thing that WarpStream provides is the control plane
- Can run in AWS, GCP and Azure
- Agents are also S3 compatible so can run with S3 compatible storage such as Minio and others
RedPanda
- Redpanda is a slimmed down native Kafka protocol compliant drop-in replacement for Kafka
- There’s even a Redpanda Connect!
- It’s main differentiator is performance, it’s cheaper and faster
Apache Pulsar
- Similar to Kafka, but changes the abstraction on storage to allow more flexibility on IO
- Has a Kafka compliant wrapper for interchangability
- Simple data offload functionality to S3 or GCS
- Multi tenancy
- Geo replication
Cloud alternatives
- Google Cloud – PubSub
- Azure – Event Hubs
- AWS – Kinesis
Tip of the Week
- Chord AI is an Android/iOS app that uses AI to figure out the chords for a song. This is really useful if you just want to get the quick jist of a song to play along with. The base version is free, and has a few different integration options (YouTube, Spotify, Apple Music Local Files for me) and it uses your phones microphone and a little AI magic to figure it out. It even shows you how to play the chords on guitar or piano. The free version gets you basic chords, but you can pay $8.99 a month to get more advanced/frequent chords.
https://www.chordai.net/ - Pandas is nearly as good, if not better than SQL for exploring data
https://pandas.pydata.org/ - Another tip for displaying in Jupyter notebooks – to HTML() your dataframes to show the full column data
https://www.geeksforgeeks.org/how-to-render-pandas-dataframe-as-html-table/ - Take photos or video and convert them into 3d models
https://lumalabs.ai/luma-api
305 פרקים
סדרה בארכיון ("עדכון לא פעיל" status)
When?
This feed was archived on March 24, 2016 19:45 (
Why? עדכון לא פעיל status. השרתים שלנו לא הצליחו לאחזר פודקאסט חוקי לזמן ממושך.
What now? You might be able to find a more up-to-date version using the search function. This series will no longer be checked for updates. If you believe this to be in error, please check if the publisher's feed link below is valid and contact support to request the feed be restored or if you have any other concerns about this.
Manage episode 425209805 series 65608


In the past couple of episodes, we’d gone over what Apache Kafka is and along the way we mentioned some of the pains of managing and running Kafka clusters on your own. In this episode, we discuss some of the ways you can offload those responsibilities and focus on writing streaming applications. Along the way, Joe does a mighty fine fill-in for proper noun pronunciation and Allen does a southern auctioneer-style speed talk.
View the full show notes here:
https://www.codingblocks.net/episode237
Reviews
As always, thank you for leaving us a review – we really do appreciate them!
From iTunes: Abucr7
Upcoming Events
Atlanta Dev Con
September 7th, 2024
https://www.atldevcon.com/
DevFest Central Florida on September 28th, 2024
Interested? Submit your talk proposal here:
https://sessionize.com/devfest-florida-orlando-2024/
Kafka Compatible and Kafka Functional Alternatives
Why? Because running any type of infrastructure requires time, knowledge, and blood, sweat and tears
Confluent
- https://www.confluent.io/confluent-cloud/pricing/
- We’ve personally had good experiences with their Kafka as a service
WarpStream
- https://www.warpstream.com/
- “WarpStream is an Apache Kafka® compatible data streaming platform built directly on top of object storage: no inter-AZ bandwidth costs, no disks to manage, and infinitely scalable, all within your VPC”
- ZERO disks to manage
- 10x cheaper than running Kafka
- Agents stream data directly to and from object storage with no buffering on local disks and no data tiering.
- Create new serverless “Virtual Clusters” in our control plane instantly
- Support different environments, teams, or projects without managing any dedicated infrastructure
- Things you won’t have to do with WarpStream
- Upscale a cluster that is about to run out of space
- Figure out how to restore quorum in a Zookeeper cluster or Raft consensus group
- Rebalance partitions in a cluster
- “WarpStream is protocol compatible with Apache Kafka®, so you can keep using all your favorite tools and software. No need to rewrite your application or use a proprietary SDK. Just change the URL in your favorite Kafka client library and start streaming!”
- Never again have to choose between reliability and your budget. WarpStream costs the same regardless of whether you run your workloads in a single availability zone, or distributed across multiple
- WarpStream’s unique cloud native architecture was designed from the ground up around the cheapest and most durable storage available in the cloud: commodity object storage
- WarpStream agents use object storage as the storage layer and the network layer, side-stepping interzone bandwidth costs entirely
- Can be run in BYOC (bring your own cloud) or in Serverless
- BYOC – you provide all the compute and storage – the only thing that WarpStream provides is the control plane
- Data never leaves your environment
- Serverless – fully managed by WarpStream in AWS – will automatically scale for you even down to nothing!
- BYOC – you provide all the compute and storage – the only thing that WarpStream provides is the control plane
- Can run in AWS, GCP and Azure
- Agents are also S3 compatible so can run with S3 compatible storage such as Minio and others
RedPanda
- Redpanda is a slimmed down native Kafka protocol compliant drop-in replacement for Kafka
- There’s even a Redpanda Connect!
- It’s main differentiator is performance, it’s cheaper and faster
Apache Pulsar
- Similar to Kafka, but changes the abstraction on storage to allow more flexibility on IO
- Has a Kafka compliant wrapper for interchangability
- Simple data offload functionality to S3 or GCS
- Multi tenancy
- Geo replication
Cloud alternatives
- Google Cloud – PubSub
- Azure – Event Hubs
- AWS – Kinesis
Tip of the Week
- Chord AI is an Android/iOS app that uses AI to figure out the chords for a song. This is really useful if you just want to get the quick jist of a song to play along with. The base version is free, and has a few different integration options (YouTube, Spotify, Apple Music Local Files for me) and it uses your phones microphone and a little AI magic to figure it out. It even shows you how to play the chords on guitar or piano. The free version gets you basic chords, but you can pay $8.99 a month to get more advanced/frequent chords.
https://www.chordai.net/ - Pandas is nearly as good, if not better than SQL for exploring data
https://pandas.pydata.org/ - Another tip for displaying in Jupyter notebooks – to HTML() your dataframes to show the full column data
https://www.geeksforgeeks.org/how-to-render-pandas-dataframe-as-html-table/ - Take photos or video and convert them into 3d models
https://lumalabs.ai/luma-api
305 פרקים
Todos os episódios
×

1 Things to Know when Considering Multi-Tenant or Multi-Threaded Applications


1 Two Water Coolers Walk Into a Bar…




1 AI, Blank Pages, and Client Libraries…oh my!


1 Alternatives to Administering and Running Apache Kafka






1 StackOverflow AI Disagreements, Kotlin Coroutines and More


1 Llama 3 is Here, Spending Time on Environmental Setup and More


1 Ktor, Logging Ideas, and Plugin Safety


1 Importance of Data Structures, Bad Documentation and Comments and More




1 Multi-Value, Spatial, and Event Store Databases


1 Overview of Object Oriented, Wide Column, and Vector Databases


1 Picking the Right Database Type – Tougher than You Think


1 There is still cool stuff on the internet


1 Reflecting on 2023 and Looking Forward to 2024


1 Gartner Top Strategic Technology Trends 2024


1 2023 Holiday Season Developer Shopping List




1 Open Telemetry – Instrumentation and Metrics


1 Keyboards, Cloud Costs, Static Analysis, and Philosophy


1 Code Confidence using NASA’s Ten Simple Rules


1 Tracing Specifics – Know your System with OpenTelmetry






1 Team Leadership, TUIs, and AI Lawsuits


1 Better Application Management with Custom Apps


1 Errors vs Exceptions, Reddit Rebels, and the 2023 StackOverflow Survey


1 Easy and Cheap AI for Developers, Reddit API Changes and Sherlocking 1:55:36


1 Gitlab vs Github, AI vs Microservices


1 Supporting Your Code, README vs Wiki and Test Coverage 1:16:35


1 Understanding Serial Transactions for Databases like Redis


1 Designing Data-Intensive Applications – Lost Updates and Write Skew


1 ChatGPT and the Future of Everything


1 Designing Data-Intensive Applications – Weak Isolation and Snapshotting


1 Designing Data-Intensive Applications – Multi-Object Transactions


1 Designing Data-Intensive Applications – Transactions




1 Job Hopping and Favorite Dev Books


1 Technical Challenges of Scale at Twitter






1 Git from the Bottom Up – Reset, Stash, and Reflog


1 Git from the Bottom Up – The Index






1 Git from the Bottom Up – Blobs and Trees




1 Site Reliability Engineering – More Evolution of Automation


1 Site Reliability Engineering – Evolution of Automation


1 Site Reliability Engineering – (Still) Monitoring Distributed Systems


1 Site Reliability Engineering – Monitoring Distributed Systems


1 Site Reliability Engineering – Eliminating Toil


1 Site Reliability Engineering – Service Level Indicators, Objectives, and Agreements


1 Site Reliability Engineering – Embracing Risk


1 Software Reliability Engineering – Hope is not a strategy




1 Minimum Viable Continuous Delivery


1 PagerDuty’s Security Training for Engineers, The Dramatic Conclusion


1 PagerDuty’s Security Training for Engineers, Penultimate


1 PagerDuty’s Security Training for Engineers! Part Deux


1 PagerDuty’s Security Training for Engineers




1 Designing Data-Intensive Applications – Secondary Indexes, Rebalancing, Routing


1 Designing Data-Intensive Applications – Partitioning






1 Transactions in Distributed Systems


1 Docker Licensing, Career and Coding Questions


1 Why Get Into Competitive Programming?




1 2021 State of the Developer Ecosystem




1 Designing Data-Intensive Applications – Leaderless Replication


1 Designing Data-Intensive Applications – Multi-Leader Replication


1 Designing Data-Intensive Applications – Single Leader Replication












1 Our Favorite Developer Tools of 2020




1 The DevOps Handbook – Create Organizational Learning




1 The DevOps Handbook – Enable Daily Learning 1:52:56


1 The DevOps Handbook – The Value of A/B Testing 1:50:13


1 Is Kubernetes Programming? 1:42:15


1 The DevOps Handbook – Enabling Safe Deployments 1:36:58


1 The DevOps Handbook – Anticipating Problems 1:22:25


1 The DevOps Handbook – The Technical Practices of Feedback 1:51:11


1 The DevOps Handbook – Architecting for Low-Risk Releases 2:05:23


1 The DevOps Handbook – The Technical Practices of Flow 1:50:51


1 Survey Says … 2020 Stack Overflow Developer Survey 2:07:41


1 Google’s Engineering Practices – How to Navigate a Code Review 1:42:16


1 Google’s Engineering Practices – What to Look for in a Code Review 1:41:48


1 Google’s Engineering Practices – Code Review Standards 1:39:01


1 Big Data – How Far is Too Far? 1:51:10


1 Designing Data-Intensive Applications – To B-Tree or not to B-Tree 1:56:40


1 How to Work Remote Effectively 2:10:24


1 Designing Data-Intensive Applications – SSTables and LSM-Trees 1:38:11


1 Designing Data-Intensive Applications – Storage and Retrieval 2:15:58


1 Why Attend Developer Conferences and What were the Hot Topics at NDC London 2020? 1:16:32


1 Designing Data-Intensive Applications – Data Models: Query Languages 1:38:44


1 Designing Data-Intensive Applications – Data Models: Relationships 2:14:00


1 Designing Data-Intensive Applications – Data Models: Relational vs Document 1:53:53


1 Designing Data-Intensive Applications – Maintainability 2:06:42


1 Designing Data-Intensive Applications – Scalability 1:52:19


1 Designing Data-Intensive Applications – Reliability 2:02:01


1 Developer Shopping Spree 2019 2:32:52


1 DevOps: Job Title or Job Responsibility? 2:00:06


1 3factor app – Async Serverless 1:31:37


1 3factor app – Reliable Eventing 2:03:13


1 3factor app – Realtime GraphQL 2:21:42


1 The Pragmatic Programmer – How to Build Pragmatic Teams 2:07:45


1 The Pragmatic Programmer – How to use Exceptions 1:54:00


1 The Pragmatic Programmer – How to Generate Code 2:02:51


1 The Pragmatic Programmer – How to Debug 1:53:30


1 The Pragmatic Programmer – Know Thy Tools 2:03:32


1 The Pragmatic Programmer – How to Estimate 1:58:10


1 The Pragmatic Programmer – Tracer Bullets and Prototyping 1:45:34


1 The Pragmatic Programmer – Is Your Code Orthogonal? 2:01:23


1 The Pragmatic Programmer – The Evils of Duplication 1:46:04


1 The Pragmatic Programmer – Investing in Your Knowledge Portfolio 2:36:43


1 Should Your Web App be a Progressive Web App (PWA)? 1:58:28


1 The Second Date is Always Easier 1:38:01


1 Why Date-ing is Hard 1:46:45


1 What Should You Learn Next? 1:14:30


1 Tackling Tough Developer Questions 1:44:15




1 Data Structures – Heaps and Tries 1:28:06


1 Data Structures – (some) Trees 1:49:56


1 Data Structures – Hashtable vs Dictionary 1:49:08


1 Data Structures – Arrays and Array-ish 2:38:38


1 Data Structures – Primitives 2:42:57


1 Developer Shopping Spree 2:43:37


1 Azure Functions and CosmosDB from MS Ignite 1:07:08


1 How to Learn Programming Skills 1:12:19


1 Comparing Git Workflows 1:48:53


1 Does Big O Matter? 1:52:13


1 What is Algorithmic Complexity? 1:41:50




1 Lightning Talks 1:58:50


1 Graph Algorithms 1:15:18


1 Algorithms You Should Know 2:15:22


1 Search Driven Apps 2:18:37


1 Programmer Strengths and Weaknesses 1:50:19


1 Understanding Complexity Theory 1:51:15




1 Design Anti-Patterns: YoYo, The God Object and More 1:28:40


1 Deliberate Practice for Programmers 2:16:51
ברוכים הבאים אל Player FM!
Player FM סורק את האינטרנט עבור פודקאסטים באיכות גבוהה בשבילכם כדי שתהנו מהם כרגע. זה יישום הפודקאסט הטוב ביותר והוא עובד על אנדרואיד, iPhone ואינטרנט. הירשמו לסנכרון מנויים במכשירים שונים.