Ctrl A
Managers ask analysts for every number about their own business and wait. Letting an LLM write SQL from scratch is fast but unreliable on large schemas: the query runs and returns a plausible wrong number. Our users ask in Azerbaijani. The system maps the question to one of 26 verified metrics. The LLM fills only allow-listed filters, groups and periods. A deterministic compiler then builds read-only SQL. The result is a self-service answer with a summary and chart, without free-form SQL.
Preliminary score
- Value for the user19.0 / 25
A specific problem and a clear outcome from solving it.
- Prototype and use of AI23.0 / 30
A working core scenario, and what AI actually contributes to the result.
- Quality testing15.0 / 20
Tests, examples of failures, and comparison with the current approach.
- Feasibility8.0 / 15
Data requirements, running costs and a clear next step.
- Originality6.0 / 10
A meaningful difference from common solutions.
What the judges wrote
- Claude Opus 5.5Preliminary68/100
- Strengths
- Your core design choice is well argued and well built: the LLM never writes SQL. It only picks one of 26 verified metrics and fills allow-listed parameters, and a deterministic compiler produces read-only DuckDB SQL. Your failure log is specific and honest. The silent all-history sum that inflated numbers about 8x, the snapshot double-count (20.50 bn vs 2.15 bn), the broken period comparison and the retrieval misrank with actual scores each show real acceptance testing with a domain expert, followed by targeted fixes. The README documents the full agent pipeline, policy validation, repair loop and Vercel deployment, which makes the prototype look genuinely end to end for Azerbaijani-language questions.
- To improve
- Your accuracy claims (98–100%) come from the dbt and Snowflake benchmarks, not from your own system. Build a labelled set of, say, 50 Azerbaijani questions and report your own routing and answer accuracy against a free-form text-to-SQL baseline on the same data. Name the adoption metric the track asks for, such as analyst hours saved or time-to-answer compared with today's analyst queue, and estimate the LLM cost per query. Replace the illustrative 128M / +12% slides and the team placeholders with real screenshots from your running product, and state a concrete next step, such as a pilot with a bank team on real deposit data.
- GPT-6 AstraPreliminary74/100
- Strengths
- Your 26-metric deposit workflow gives Azerbaijani-speaking managers a clearly bounded self-service analytics use case. Your allow-listed parameter inference and deterministic compiler make AI useful without trusting generated SQL. Your reported failures—especially snapshot double-counting and missing default periods—are unusually specific, with concrete corrections and DuckDB-backed regression tests.
- To improve
- Your next evaluation should use held-out Azerbaijani questions with live LLM calls, reporting routing, parameter and answer accuracy against analyst-verified answers and a free-form SQL baseline. Your cited external benchmark percentages do not establish this prototype’s accuracy; include an actual question-to-chart trace with expected and returned values. Your pilot plan should specify data onboarding, access controls, per-query cost and latency, and measure analyst turnaround time saved. Your regression suite should also demonstrate that unsupported questions cannot fall back to dynamic SQL.
The team
Khatira AliyevaLead
developer
Tahmina Karimova
developer
Maarif Sariyev
data