Docs as Tests: Keeping documentation resilient to product changes with Manny Silva

The Not-Boring Tech Writer

In this episode, I'm talking with Manny Silva, a technical writer who created the "Docs as Tests" concept name and the open-source tool Doc Detective. We discuss how to automatically test your documentation for accuracy, why customer reports of broken docs are actually failed tests, and practical ways to implement automated documentation testing regardless of your tech stack.

Manny and I discuss his background as someone who intentionally chose technical writing as a career path, starting with early exposure to computers through his mother's work and developing into roles at Apple, Google, and now Skyflow as Head of Documentation. We explore the core concept behind Docs as Tests—that documentation contains testable assertions about how a product should work, and that customer reports of broken procedures are essentially failed tests that we should catch proactively rather than reactively.

We dive deep into how Manny's strategy works in practice, from the "cupcake to wedding cake" approach of starting small and scaling up. We dig into two different approaches to the technical implementation: creating “detected” tests using Doc Detective, which reads the docs directly and uses them as tests, and creating standalone tests in testing tools like Playwright or Cypress, which you’d create and update independently of the docs. Manny explains how his Doc Detective tool can parse markdown documentation, automatically execute the steps described in procedures, capture screenshots for visual regression testing, and even validate API responses against OpenAPI schemas. We discuss the business case for automated documentation testing, including how it prevents customer frustration, builds trust, reduces support overhead, and can catch bugs before they reach production.

Throughout our conversation, we explore practical implementation strategies, including how to sell the approach to stakeholders, integrate testing into CI/CD pipelines, handle multifactor authentication challenges, and work with QA teams. Manny also shares his philosophy of creating a "zero trust" relationship between docs and product—not out of disrespect, but to ensure everyone stays honest about the behavioral contract that documentation represents. Docs as Tests also encourages technical writers to embrace their unofficial QA role–as writers, we’re often the first to test a new feature or product, and embracing a Docs as Tests mindset can help legitimize and make visible this role.

About Manny Silva:

Technical writer by day, engineer by night, and father everywhere in between, Manny wears many (figurative) hats. He's passionate about intuitive and scalable developer experiences, and he likes diving into the deep end as the 0th user.

Here are a few things that keep him busy:

• Head of Docs at Skyflow, a data privacy vault company.
• Codifier of Docs as Tests, a tool-agnostic strategy for keeping docs and their products in sync by using doc content as product tests.
• Creator and maintainer of Doc Detective, an open-source doc testing framework.
• AI development and experimentation.

He's always looking for collaborators on projects, and he loves chatting with folks, so don't hesitate to reach out.

Resources discussed in this episode:

• Docs as Tests: A Strategy for Resilient Technical Documentation - Manny's book
• Docs as Tests blog - Manny's blog about the strategy and various tools
• Doc Detective - Manny's open source tool for testing and validating documentation
• Doc Detective GitHub action - Official GitHub action for CI/CD integration
• Doc Detective Discord server - Public community for users implementing Docs as Tests
• Good to Great book series - Business development books that Manny recommends
• Framework laptop - Repairable laptop that Manny built with his children
• Vale - Style guide enforcement tool (mentioned as complementary to Docs as Tests)
• Playwright - Engineering-level testing tool used by some companies like Docker
• Cypress - Another engineering-level testing tool
• Ben Perlmutter - Unit Test the Docs: Why You Should Test Your Code Examples - A Write the Docs Portland 2022 talk about MongoDB's unit testing documentation approach
• Arazzo specification - Newer OpenAPI initiative specification for workflow testing

—

Contact The Not-Boring Tech Writer team:

We love hearing your ideas for episode topics, guests, or general feedback:

• Email: [email protected]
• thenotboringtechwriter.com
• LinkedIn
• Bluesky
  
  

Join the discussion by replying on Bluesky

Contact Kate Mueller:

• knowledgewithsass.com
• LinkedIn
• Bluesky
  
  

Contact Manny Silva:

• Doc Detective
• LinkedIn

Contact KnowledgeOwl:

• KnowledgeOwl.com
• LinkedIn