RETRACE — AI-Assisted Visual Regression Testing for Games
Game developers and QA teams need to distinguish intentional visual changes from regressions across builds. An outfit update may be allowed, while a missing world object may indicate a bug. Pixel differences alone cannot reliably explain that distinction. RETRACE compares reference and candidate screenshots, localizes changes, and uses a vision-language model to evaluate observations against developer-defined rules. It returns PASS, FAIL, or NEEDS_REVIEW with inspectable images, crops, rules, and an evidence report. Our working demonstration identifies a missing barrel and records a FAIL with supporting visual evidence. Developers retain oversight through review and controlled reference approval. The prototype currently analyzes supplied image pairs. Our longer-term vision is a gameplay agent that repeats comparable scenarios across builds, records gameplay, and produces evidence-backed incident reports.
Preliminary score
- Value for the user18.5 / 25
A specific problem and a clear outcome from solving it.
- Prototype and use of AI23.5 / 30
A working core scenario, and what AI actually contributes to the result.
- Quality testing17.5 / 20
Tests, examples of failures, and comparison with the current approach.
- Feasibility13.0 / 15
Data requirements, running costs and a clear next step.
- Originality7.0 / 10
A meaningful difference from common solutions.
A finalist: the live jury scored it 68.0 / 100 on stage.
What the judges wrote
- Claude Opus 5.5Preliminary78/100
- Strengths
- Your problem framing is sharp: a visual difference is not a verdict, and your pipeline reflects that. DINOv2 and pixel diffs propose regions, Gemini judges each one against developer rules, and deterministic code makes the final PASS/FAIL/NEEDS_REVIEW call. Your testing is unusually honest and specific: per-pair outcomes on 12 labelled pairs, a pixel-control and Qwen baseline, a clearly documented miss (the pedestal read as lighting), and caveats about label provenance and confounding. The engineering evidence also stands out: 199 passing tests, hash-verified evidence ZIPs, a Docker build, and the line-ending hash bug you found and fixed.
- To improve
- Your evaluation set of 12 pairs is too small to show the method beats the current approach. The planned set of new, independently labelled pairs is the most important next step. With 6 of 7 clean pairs deferred to review, the reviewer workload saving is not yet shown, so measure review time against manual build comparison by a QA tester. The judges could only see a replayed recorded run, so a fresh live inference in the video or demo would strengthen the case that the core scenario works end to end. Validating the 'visual contracts' rule format with even one or two game developers would turn the asserted user value into evidence.
- GPT-6 AstraPreliminary81/100
- Strengths
- Your prototype addresses a concrete QA problem: distinguishing permitted visual changes from regressions, rather than merely detecting different pixels. Your barrel example and report view connect model observations to developer rules, paired crops, and exportable evidence. Your testing distinguishes decision coverage from accuracy, documents the missed pedestal and false alarms, and includes concrete evidence-integrity and deployment checks.
- To improve
- Your next evaluation should use independently labelled, held-out build pairs, control the source/class confounding, and compare against both manual QA and a calibrated image-difference baseline. Prioritize the incorrect pedestal PASS and the lack of any clean PASS before expanding into gameplay agents, measuring whether object-presence checks reduce misses without increasing unnecessary reviews. Your studio pilot should establish reviewer time saved and report per-method latency and full operating cost at a realistic build volume.
The team
Celal IbrahimliLead
data
Ali Agabalazade
developer