AI Evaluation and Testing: How to Know When Your Product Works (or Doesn’t)

The AI Native Dev - from Copilot today to AI Native Software Development tomorrow

Player FM - Internet Radio Done Right

0-10 subscribers

הוסף לפני one שנה

תוכן מסופק על ידי Tessl. כל תוכן הפודקאסטים כולל פרקים, גרפיקה ותיאורי פודקאסטים מועלים ומסופקים ישירות על ידי Tessl או שותף פלטפורמת הפודקאסט שלהם. אם אתה מאמין שמישהו משתמש ביצירה שלך המוגנת בזכויות יוצרים ללא רשותך, אתה יכול לעקוב אחר התהליך המתואר כאן https://he.player.fm/legal.

The Innovators & Investors Podcast

1
Inside Deloitte Ventures: Strategic Corporate VC Insights on Scaling Startups and Vertical AI Trends 34:07

לפני 16 ימים34:07

הפעל מאוחר יותר

רשימות

לייק

אהבתי

34:07

In this insightful episode of The Innovators & Investors Podcast, host Kristian Marquez sits down with Jay Crone, Managing Director and Venture Capitalist at Deloitte Ventures, to explore the firm’s strategic approach to corporate venture capital. Jay shares Deloitte Ventures’ investment thesis, focusing on supporting innovative Canadian startups at Series A and B stages across key sectors like cybersecurity, climate tech, fintech, future of work, health tech, and AI. Listeners will gain an inside look at how Deloitte leverages its vast network of 1,500 partners and 15,000 employees to source deals and add value beyond capital by helping startups navigate Deloitte’s complex ecosystem and access enterprise clients. Jay also discusses his diverse career journey—from government and investment banking to entrepreneurship and corporate VC—and how those experiences shape his investment philosophy. The episode delves into Deloitte’s due diligence process, the importance of founder relationships, and the firm’s strategic role as a co-investor. Jay highlights emerging trends, particularly the promise of vertical AI tailored to industry-specific needs, and shares his bullish outlook on fintech innovations like stablecoins and cross-border payments. He offers candid advice for entrepreneurs on risk-taking and aligning business vision with funding goals. This episode is a must-listen for founders, investors, and anyone interested in the evolving landscape of corporate venture capital and innovation in Canada. Learn more about Jay's work at https://www.deloitte.com/ca/en/services/program/ventures.html Connect with Jay on LinkedIn at https://www.linkedin.com/in/jaycrone/ Think you'd be a great guest on the show? Apply at https://finstratmgmt.com/innovators-investors-podcast/ Want to learn more about Kristian Marquez's work? Check out his website at https://finstratmgmt.com…

לפני שנה 50:40

MP3•בית הפרקים

This episode of AI Native Dev, hosted by Simon Maple and Guy Podjarny, features a mashup of conversations with leading figures in the AI industry. Guests include Des Traynor, founder of Intercom, who discusses the paradigm shift generative AI brings to product development. Rishabh Mehrotra, Head of AI at SourceGraph, emphasizes the importance of evaluation processes over model training. Tamar Yehoshua, President of Products and Technology at Glean, shares her experiences in enterprise search and the challenges of using LLMs in data-sensitive environments. Finally, Simon Last, Co-Founder and CTO of Notion, talks about continuous improvement and the iterative processes at Notion. Each guest provides invaluable insights into the evolving landscape of AI-driven products.
Watch the episode on YouTube: https://youtu.be/gZ4sGROvOdQ

Join the AI Native Dev Community on Discord: https://tessl.co/4ghikjh
Ask us questions: podcast@tessl.io

64 פרקים

#Tech #Tessl #AI Native Software Development #Ai Developments #Ai Software Development #AI Native #Ai Assistants #Ai Code Completion

AI Evaluation and Testing: How to Know When Your Product Works (or Doesn’t)

The AI Native Dev - from Copilot today to AI Native Software Development tomorrow

0-10 subscribers

published לפני שנה

שתפו

MP3•בית הפרקים

Join the AI Native Dev Community on Discord: https://tessl.co/4ghikjh
Ask us questions: podcast@tessl.io

64 פרקים

#Tech #Tessl #AI Native Software Development #Ai Developments #Ai Software Development #AI Native #Ai Assistants #Ai Code Completion

כל הפרקים

1
What If Fixing Code Wasn’t Your Job Anymore? | Jonathan Schneider & Moderne 36:37

לפני 1 day36:37

36:37

Jonathan Schneider , co-founder and CEO of Moderne , joins Simon Maple to share how real engineering teams are using Moderne’s rewrite engine to reduce technical debt at the source, and drive org-wide transformation. On the docket: the hard limit LLMs can’t scale past why OpenRewrite uses a declarative framework the real challenge: accessing the compiler’s truth AI Native Dev , powered by Tessl and our global dev community, is your go-to podcast for solutions in software development in the age of AI. Tune in as we engage with engineers, founders, and open-source innovators to talk all things AI, security, and development. Connect with us here: 1. Jonathan Schneider- https://www.linkedin.com/in/jonkschneider/ 2. Simon Maple- https://www.linkedin.com/in/simonmaple/ 3. Tessl- https://www.linkedin.com/company/tesslio/ 4. AI Native Dev- https://www.linkedin.com/showcase/ai-native-dev/ Join the AI Native Dev Community on Discord: https://tessl.co/4ghikjh Ask us questions: podcast@tessl.io…

1
Why AI Coding Agents Are Here To Stay | Patrick Debois 48:57

לפני 9 ימים48:57

48:57

Patrick Debois , the mind behind “DevOps,” joins Simon Maple to unpack the system-level shifts AI is driving across software engineering, drawn from what he saw firsthand at the AI Engineer’s World Fair. They also get into: • how inconsistent codebases confuse AI • why running agents locally is becoming obsolete • inside OpenAI’s concept of “model specs” AI Native Dev , powered by Tessl and our global dev community, is your go-to podcast for solutions in software development in the age of AI. Tune in as we engage with engineers, founders, and open-source innovators to talk all things AI, security, and development. Read Patrick Debois’s full article here: https://ainativedev.co/0p3 Connect with us here: 1. Patrick Debois- https://www.linkedin.com/in/patrickdebois/ 2. Simon Maple- https://www.linkedin.com/in/simonmaple/ 3. Tessl- https://www.linkedin.com/company/tesslio/ 4. AI Native Dev- https://www.linkedin.com/showcase/ai-native-dev/ (00:00) Trailer (01:03) Introduction (02:57) AI Engineering World Fair: Conference Overview (07:38) Seven Key Lessons from the Fair (08:51) AI Coding Agents Everywhere (11:14) Evolving AI Dev Tool Practices (12:23) Specs as the New Code (15:00) Agents Running in the Cloud (24:00) Parallel Agent Execution (36:58) CI/CD Shifting Left (40:55) AI Productivity Multipliers (47:15) Outro Join the AI Native Dev Community on Discord: https://tessl.co/4ghikjh Ask us questions: podcast@tessl.io…

1
The Graph Layer Behind NASA’s Breakthroughs | Michael Hunger 36:24

לפני 16 ימים36:24

36:24

Michael Hunger of Neo4j, joins Simon Maple to unpack how graph databases inject structure, intent, and traceability into modern AI systems. On the docket: why relationships in data encode intent the black-box problem in vector based RAG why devs should build their own MCP server AI Native Dev, powered by Tessl and our global dev community, is your go-to podcast for solutions in software development in the age of AI. Tune in as we engage with engineers, founders, and open-source innovators to talk all things AI, security, and development. Connect with us here: Michael Hunger- https://www.linkedin.com/in/jexpde/ Simon Maple- https://www.linkedin.com/in/simonmaple/ Tessl- https://www.linkedin.com/company/tesslio/ AI Native Dev- https://www.linkedin.com/showcase/ai-native-dev/ (00:00) Trailer (01:03) Introduction & Neo4j Origins (03:02) Persisting Relationships for High-Performance Queries (04:00) Modeling Business Intent & Key Use Cases (05:00) Fraud Detection at Scale with Graph Algorithms (06:11) Graph-Enhanced RAG vs. Vector-Only Retrieval (09:02) Explainability & Drill-Down Evaluation in RAG (13:05) Fusing Structured & Unstructured Data for Context (15:00) MCP for Developer Productivity: Schema-to-Code & API Wrapping (21:16) Security & Sandboxing Best Practices for MCP (29:08) MCP Server Recommendations & Outro Join the AI Native Dev Community on Discord: https://tessl.co/4ghikjh Ask us questions: podcast@tessl.io…

1
The Missing Gap In Workflows For AI Devs | Baruch Sadogursky 49:00

לפני 23 ימים49:00

49:00

Baruch Sadogursky, Head of Developer Relations at TuxCare, joins Simon Maple to explore why automated integrity needs to be built in before we rely on AI outputs. On the docket: the difference between specs and tests why PMs sidelined specs the "intent-integrity gap" between human goals and LLM outputs the non-determinism of LLMs as a feature, not a flaw Baruch’s belief: devs are not going anywhere AI Native Dev, powered by Tessl and our global dev community, is your go-to podcast for solutions in software development in the age of AI. Tune in as we engage with engineers, founders, and open-source innovators to talk all things AI, security, and development. Connect with us here: 1. Baruch Sadogursky- https://www.linkedin.com/in/jbaruch/ 2. Simon Maple- https://www.linkedin.com/in/simonmaple/ 3. Tessl- https://www.linkedin.com/company/tesslio/ 4. AI Native Dev- https://www.linkedin.com/showcase/ai-native-dev/ Join the AI Native Dev Community on Discord: https://tessl.co/4ghikjh Ask us questions: podcast@tessl.io…

1
Why LLMs Keep Missing This One Thing | Jason Ganz 49:20

לפני 30 ימים49:20

49:20

Can LLMs replace structured systems to scale enterprises? Jason Ganz, Senior Manager DX at dbt Labs , joins Simon Maple to unpack why, despite the rapid rise of AI systems, enterprises still rely on structured data for consistency and reliable decision making. They also discuss: the invisible edge cases LLMs can’t see difference between software engineering and data engineering in AI the mismatch between AI output and business logic what the data engineer of the future actually does AI Native Dev , powered by Tessl and our global dev community, is your go-to podcast for solutions in software development in the age of AI. Tune in as we engage with engineers, founders, and open-source innovators to talk all things AI, security, and development. Connect with us here: 1. Jason Ganz (LinkedIn)- https://www.linkedin.com/in/jasnonaz/ 2. Jason Ganz (X)- https://x.com/jasnonaz 3. dbt Labs- https://www.getdbt.com/ 4. dbt Fusion engine- https://www.getdbt.com/product/fusion 5. dbt Community- https://www.getdbt.com/community 6. Simon Maple- https://www.linkedin.com/in/simonmaple/ 7. Tessl- https://www.linkedin.com/company/tesslio/ 8. AI Native Dev- https://www.linkedin.com/showcase/ai-native-dev/ 00:00 Trailer 01:01 Introduction 01:41 dbt Labs 04:39 Data engineers 07:39 LLMs understanding 13:15 AI isn’t as lazy as humans 15:29 Problem: the scaffolding to get data 17:38 Best contextual results 19:40 Dealing with security 25:00 Structured data 27:37 Problems with LLMs and data 29:47 Exact numbers 32:10 Hallucinations 34:28 Human validation 36:20 MCP servers 39:09 UX bottlenecks 42:27 Quality of data 44:00 The future of data engineers 47:02 getdbt.com 48:09 Outro Join the AI Native Dev Community on Discord: https://tessl.co/4ghikjh Ask us questions: podcast@tessl.io…

1
CTO of $7B Snyk Talks AI Security, Risky Software & Enterprise Adoption 35:43

לפני 5 weeks35:43

35:43

In this episode of AI Native Dev, Guy Podjarny and Danny Allan unpack how security has reduced just to a concern from a roadblock for devs. On the docket: why 80% of Snyk ’s enterprise customers are actively using AI tools navigating security risks of today and tomorrow the recurring flaw in every new stack why more code means more vulnerabilities AI Native Dev , powered by Tessl and our global dev community, is your go-to podcast for solutions in software development in the age of AI. Tune in as we engage with engineers, founders, and open-source innovators to talk all things AI, security, and development. Connect with us here: Danny Allan- https://www.linkedin.com/in/dannyallan/ Guy Podjarny- https://www.linkedin.com/in/guypo/ Tessl- https://www.linkedin.com/company/tesslio/ AI Native Dev- https://www.linkedin.com/showcase/ai-native-dev/ Join the AI Native Dev Community on Discord: https://tessl.co/4ghikjh Ask us questions: podcast@tessl.io…

1
The $2B Nvidia Backed AI Video Platform 1:11:11

לפני 6 weeks1:11:11

1:11:11

In this episode of AI Native Dev, Guy Podjarny is joined by Victor Riparbelli , Co-founder and CEO of Synthesia , who is leading the charge towards AI driven video that lets anyone become a video creation pro with no prior field experience. On the docket: • how Synthesia enables full video editability even after generation • Victor’s vision for fully coded videos that enable personalized, customizable viewing • Victor’s belief that future generations will prefer visuals over any kind of text AI Native Dev , powered by Tessl and our global dev community, is your go-to podcast for solutions in software development in the age of AI. Tune in as we engage with engineers, founders, and open-source innovators to talk all things AI, security, and development. Connect with us here: 1. Victor Riparbelli- https://www.linkedin.com/in/victorriparbelli/ 2. Guy Podjarny- https://www.linkedin.com/in/guypo/ 3. Tessl- https://www.linkedin.com/company/tesslio/ 4. AI Native Dev- https://www.linkedin.com/showcase/ai-native-dev/ Join the AI Native Dev Community on Discord: https://tessl.co/4ghikjh Ask us questions: podcast@tessl.io…

1
Why the Top 1% of Devs Love IntelliJ | Anton Arhipov 36:59

לפני 7 weeks36:59

36:59

From early IntelliJ user to JetBrains advocate, Anton Arhipov continues his conversation with Simon Maple and Baptiste Fernandez on AI Native Dev, tracing his journey through tools, trends, and the changing face of developer experience. On the docket: • how IntelliJ responds to new tools in the market • understanding the pros and cons of tab-driven development for devs • why tools will need to get better in PR workflows • ZeroTurnaround’s Java reloading that ended JVM restarts • the four must-know IntelliJ plugins for every developer AI Native Dev, powered by Tessl and our global dev community, is your go-to podcast for solutions in software development in the age of AI. Tune in as we engage with engineers, founders, and open-source innovators to talk all things AI, security, and development. Connect with us here: 1. Anton Arhipov- https://www.linkedin.com/in/antonarhipov/ 2. Simon Maple- https://www.linkedin.com/in/simonmaple/ 3. Baptiste Fernandez - https://www.linkedin.com/in/baptiste-fernandez-0a958630/ 4. Tessl- https://www.linkedin.com/company/tesslio/ 5. AI Native Dev- https://www.linkedin.com/showcase/ai-native-dev/ Join the AI Native Dev Community on Discord: https://tessl.co/4ghikjh Ask us questions: podcast@tessl.io…

1
How This Broadcom Dev Uses Spring to Power AI Workflows | Josh Long 44:01

לפני 8 weeks44:01

44:01

What happens when the OG Spring Dev, Josh Long , teams up with Simon Maple on AI Native Dev? You get a front row seat to them building a Spring powered app that showcases the future of AI integration. While building the app, they discuss: why developers are adopting Spring AI how Spring is becoming the go-to for AI engineering why companies route users from humans to IVRs understanding model context protocol AI Native Dev , powered by Tessl and our global dev community, is your go-to podcast for solutions in software development in the age of AI. Tune in as we engage with engineers, founders, and open-source innovators to talk all things AI, security, and development. Connect with us here: Josh Long- https://www.linkedin.com/in/joshlong/ Simon Maple- https://www.linkedin.com/in/simonmaple/ Tessl- https://www.linkedin.com/company/tesslio/ AI Native Dev- https://www.linkedin.com/showcase/ai-native-dev/ Join the AI Native Dev Community on Discord: https://tessl.co/4ghikjh Ask us questions: podcast@tessl.io…

1
One CEO’s Plan to Rebuild How We Deploy AI | Ron Eforni & Flox 1:03:30

לפני 9 weeks1:03:30

1:03:30

In this episode, our founder and CEO, Guy Podjarny speaks with Ron Efroni, CEO of Flox and President of the NixOS Foundation, about advancing AI from experimental phases to robust, production-ready environments. Flox optimizes reproducible environments, Nix provides the foundation, and Ron Efroni is uniting them to scale enterprise development. On the docket: the value proposition of using "environment as code" exploring a future of specialized AI agents in a connected ecosystem. grasping how to layer MCP over environments, and AI to be more deterministic, or should we adapt to it? About Us: AI Native Dev, powered by Tessl and our global dev community, is your go-to podcast for solutions in software development in the age of AI. Tune in as we engage with engineers, founders, and open-source innovators to talk all things AI, security, and development. Connect with us here: Ron Efroni- https://www.linkedin.com/in/ronefroni/ Guy Podjarny- https://www.linkedin.com/in/guypo/ Tessl- https://www.linkedin.com/company/tesslio/ AI Native Dev- https://www.linkedin.com/showcase/ai-native-dev/ Join the AI Native Dev Community on Discord: https://tessl.co/4ghikjh Ask us questions: podcast@tessl.io…

1
Vibe Coding SimCity II: Injecting Chaos with Natural Disasters and AI Tools 1:00:42

לפני 10 weeks1:00:42

1:00:42

Simon Maple and Joe Holdcroft swap their AI-generated apps and put Cursor and Windsurf to the test by adding natural disasters like earthquakes and lightning strikes. Discover how AI coding tools handle randomness, spreading effects, animation, and bugs—plus the devs’ thoughts on debugging AI-generated code. Subscribe for deep dives into modern dev workflows and AI-powered coding experiments. Watch the episode on YouTube: https://youtu.be/dzXGY8YUx-8 Join the AI Native Dev Community on Discord: https://tessl.co/4ghikjh Ask us questions: podcast@tessl.io…

1
Vibe Coding SimCity: Prototyping Tiny Towns with AI Dev Tools 44:00

לפני 11 weeks44:00

44:00

In this hands-on episode of AI Native Dev , Simon Maple and Joe Holdcroft go head-to-head in a “vibe coding relay” challenge—building a tiny town simulator from scratch using AI-powered dev tools BASE44 and Bolt.new. With only 10 minutes, vague prompts, and no prep, they put these rapid-prototyping tools to the test. Simon leans into SimCity nostalgia while Joe battles ghost-town bugs and infinite money glitches. Hosted by Macey Baker, the episode dives deep into AI-assisted UI generation, event simulation, and how far these tools can really go with minimal human input. Part 2 continues next week with Cursor and Windsurf as Simon and Joe swap codebases to iterate on each other’s AI-born towns. Join the AI Native Dev Community on Discord: https://tessl.co/4ghikjh Ask us questions: podcast@tessl.io…

1
Exploring LLM Observability with Traceloop's Gal Kleinman 40:26

לפני 12 weeks40:26

40:26

In this episode, Simon Maple converses with Gal Kleinman, co-founder of Traceloop, to delve into the realm of LLM observability. Kleinman shares his expertise on constructing evaluation suites and overcoming the challenges associated with LLM applications. Learn about innovative solutions like Open LLMetry and gain insights into best practices for maintaining robust observability systems. Subscribe to the AI Native Developer podcast for more insights on AI and development. Watch the episode on YouTube: https://youtu.be/TD-wjNStTVM Check out Traceloop: https://www.traceloop.com/ Join the AI Native Dev Community on Discord: https://tessl.co/4ghikjh Ask us questions: podcast@tessl.io…

1
From Builder to Orchestrator—Confronting the Software Engineer’s Identity Crisis 54:37

לפני 13 weeks54:37

54:37

In this episode, host Simon Maple engages in a compelling discussion with Annie Vella, a skilled software engineer and manager, about the integration of AI into software development. Annie shares her insights on how AI tools are reshaping coding practices, the challenges engineers face in transitioning to management roles, and the skills required to thrive in this evolving landscape. This episode is crucial for anyone looking to understand the implications of AI in their work. Subscribe to the AI Native Developer podcast for more insights on AI and development. Watch the episode on YouTube: https://youtu.be/GI2ywMl2gQk Read Annie's blog : https://annievella.com/posts/the-software-engineering-identity-crisis/ Join the AI Native Dev Community on Discord: https://tessl.co/4ghikjh Ask us questions: podcast@tessl.io…

1
Is Code Dead & The $1B Solo Startup Myth - 5 AI Realities with Tessl's Guy Podjarny 29:03

לפני 14 weeks29:03

29:03

In this episode, Guy Podjarny sits down with Tom Hulme to discuss the transformative shift from a code-centric to a spec-centric approach in software development. They explore the advantages of cloud-native practices, the evolving tool ecosystem, and the future role of AI and LLMs. Gain expert insights into the trends that are shaping the future of the industry. Subscribe to the AI Native Developer podcast for more insights on AI and development! Watch the episode on YouTube: https://youtu.be/reU8W2aH30Y Read the GV Event Blog: gv.com/news/five-ai-realities-with-tessl-guy-podjarny Join the AI Native Dev Community on Discord: https://tessl.co/4ghikjh Ask us questions: podcast@tessl.io…

ברוכים הבאים אל Player FM!

Player FM סורק את האינטרנט עבור פודקאסטים באיכות גבוהה בשבילכם כדי שתהנו מהם כרגע. זה יישום הפודקאסט הטוב ביותר והוא עובד על אנדרואיד, iPhone ואינטרנט. הירשמו לסנכרון מנויים במכשירים שונים.