Colvin Run was building a critical application for the United States Air Force. Given the mission-critical nature, establishing thorough end-to-end testing was a priority from day one.
Before Cofactor, testing meant engineers clicking through the app by hand: slow and inconsistent. Even with manual QA, bugs would surface up late in the dev process, requiring context switching for developers.
"As an engineering leader, you're constantly weighing coverage against velocity," says Tom Houman, VP of Engineering. Tom's goal was to find a testing solution that would help build core coverage without sacrificing their team's velocity.
Tom understood the heavy engineering cost of maintaining traditional E2E automations. On the other hand, he saw that AI agents can be unreliable and he wanted auditability and reliability in their test suite. He chose Cofactor due to the product's ability to flexibly navigate UI's, while producing a repeatable, deterministic and self-healing Playwright suite.
Colvin Run adopted Cofactor for Agentic E2E testing. Within two weeks, the development team had core coverage established for their product.
Engineers write tests from the CLI, inside their own codebase, alongside their coding agents, with no separate tooling to context-switch into. New features are pushed to a staging env and Cofactor tests run overnight; the same flows run locally on demand while a feature is still being built.
Unlike traditional E2E tests, Cofactor also tests core AI features within the application such as PDF generation and export. Ensuring these AI features worked correctly previously required lengthy manual QA.
30+ engineer-hours saved per month. Colvin Run estimates 2+ hours saved daily from the nightly and on-demand feedback loop that the Cofactor tests provide.
Faster feature development. Since tests run continuously, the team can develop more quickly without feeling worried about breaking core flows. As a Senior Engineer puts it: "I'm able to add new features without worrying about breaking the current product. Whenever I'm building a new feature, I'm not clicking through the whole app to make sure I'm not breaking it."
Core flows covered in under two weeks, with additional coverage built.Colvin Run is now leveraging Cofactor to build and maintain agentic E2E tests for two additional products.
Near-zero onboarding. A teammate returning to the project repo after weeks away ran the suite with no guidance beyond "ask Claude to test the new feature locally." As a Senior Engineer put it: "Cofactor is more powerful than Claude + Playwright. It makes it much easier to build up and stand up a regression suite with minimal training."
Cleaner demos. Breakage that once showed up live now gets caught much earlier. "I can be more confident that it's going to work when I'm giving a demo to someone, because I see my test suites all green."
“Cofactor has freed our engineers from the grind of writing and maintaining tests. They're spending that time on the work that actually moves the product forward. It's been a real force multiplier.”