How Woven by Toyota Built a 10x Faster Bug Triage Agent for Autonomous Driving

Woven by Toyota is the autonomous driving and mobility technology arm of Toyota Motor Corporation, responsible for developing the software systems that enable vehicles to perceive, plan, and navigate the real world safely. For Suigen Koide, Head of DevBoost and Automated Driving at Woven by Toyota, this mission is personal: struck by a truck at four years old, he recovered after four days unconscious and has dedicated more than eight years to building systems that could have prevented that accident. With over 1.3 million people dying in road accidents annually worldwide, the stakes for his team are not abstract.

Every autonomous driving bug they detect generates a workflow: collect the driving log video from test vehicles, classify the failure into the correct subcategory, route it to the right engineering team, iterate, test on a closed course, and deploy a fix over-the-air. The bottleneck that kept surfacing was triage — the manual process of watching footage frame-by-frame, identifying what went wrong, and classifying it correctly across many granular subcategories. Small distinctions mattered enormously: misclassifying "unable to detect a green traffic light" versus "lost perception of green traffic light" would send the bug to the wrong team and compound the delay.

The team set out to build AutoTriage, a video AI agent that would replace manual triage with automated classification. The initial version appeared promising but quickly revealed its limitations: the team was feeding low-resolution thumbnail images rather than actual driving log footage, prompts lacked the domain precision needed for nuanced categories, and the multimodal models available at the time (including early versions of GPT-4o and Gemini) struggled with video inputs beyond ten frames. Experiments were scattered across Word documents and PowerPoint files with no structured tracking or visualization. The team spent a year iterating without meaningful accuracy gains.

Four changes broke the deadlock. The team rebuilt their data pipeline to feed high-resolution driving video directly to the model. Domain experts were recruited into the project to co-write prompts grounded in actual triage knowledge. Gemini 2.5 Pro was released and unlocked the level of detailed video reasoning the task required. And W&B Weave was adopted as the central hub for experiment tracking and evaluation — with the W&B team adding video support to Weave specifically for this use case. Weave gave the team a single place to log all experiments, visualize inputs and outputs side-by-side, and measure weighted precision and recall across iterations rather than hunting through static documents.

With all four improvements in place, AutoTriage achieved high accuracy and delivered a 10x increase in triage speed and scale with the same team and resources. The next milestone is a multi-agent architecture: AutoTriage classifies initial bugs, then a Triager agent cross-references them against existing bugs using expert-made explanation videos, moving toward AI-assisted triage that can scale well beyond the ceiling of human capacity.