AI Enterprise SolutionsGPT & Claude

Fonim

Teams building AI agents on open-weight models: startups, and companies in regulated sectors that can't send data to a cloud API. Out of the box, small open models fail at real web tasks (ours solved 0 of 21). Improving them takes weeks of manual work across evaluation, sandboxing, data collection and LoRA training, with no proof the new version is better.

Video GitHub

Preliminary score

66.0/ 100Promisingaverage of 2 judges

Value for the user17.5 / 25
A specific problem and a clear outcome from solving it.
Prototype and use of AI15.5 / 30
A working core scenario, and what AI actually contributes to the result.
Quality testing16.0 / 20
Tests, examples of failures, and comparison with the current approach.
Feasibility10.5 / 15
Data requirements, running costs and a clear next step.
Originality6.5 / 10
A meaningful difference from common solutions.

What the judges wrote

Claude Opus 5.5Preliminary63/100
Strengths
Your failure reporting is unusually honest and specific: 0 of 21 tasks solved by real Qwen3-VL-2B, invalid list-format coordinates, a prompt fix that cut invalid actions to 51%, and a QLoRA run that halved loss without changing behaviour. You also named the cold-start flaw in your own loop and proposed seed demonstrations as the fix. Encoding security rules as tests (localhost-only API, safetensors-only, held-out isolation, no auto-activation, sandboxed traffic) is a thoughtful design choice for regulated-sector users. Running a real 2B vision model in 4-bit on a 6 GB RTX 3050 shows the local-first promise is grounded.
To improve
Your core claim, that the lab turns v1 into a measurably better v2, is not demonstrated: the only success numbers come from a scripted mock model, and the real model never produced a v2. Your README still marks phases 3–8 (sandbox, evaluator, UI, trainer, reports) as unchecked, which contradicts the submission, so update it. Also, the setup text refers to a demo link while none was provided. Add a comparison with today's manual process, such as hours of engineer time per improvement cycle. Also state running costs (GPU hours, energy) per evolution cycle, and show even one small v1→v2 gain using seeded demonstrations.
GPT-6 AstraPreliminary69/100
Strengths
Your local-first workflow addresses a concrete need for teams that cannot send data to cloud APIs. Your testing account is unusually informative: 0/21 real-model successes, malformed actions, and falling training loss without behavioral improvement expose the actual bottlenecks. Your held-out task separation, manual activation, and mock-only network access are sensible safeguards for iterative agent training.
To improve
Your next milestone should use labelled seed demonstrations to produce a real v2 and compare it with v1 on untouched tasks; scripted MockLAM evaluation does not establish that the improvement loop works. Your README marks the evaluator, sandbox, training, and reporting as unfinished, so reconcile that checklist with your submission and provide reproducible run reports. Your adoption case also needs measured training/evaluation time, running costs, and a comparison with the current manual workflow.

The team

Seid CeferzadeLead
product
Nəsir Sadıxov
data

Disclosure

Required by the rules. Judges verify it against the repository.

Models: Qwen3-VL-2B, run locally in 4-bit, MockLAM, Cloude Code
Data: Synthetic only. 3 mock websites (shop, bank, mail) with fake data and 30 tasks, all written by us. Training data comes from the model's own episodes on these sites. No real user data.
Components: FastAPI, SQLModel/SQLite, Pydantic, Typer; React + TypeScript + Vite; Playwright (Chromium); PyTorch, Transformers, PEFT, bitsandbytes; Cloudflare Tunnel.
Built during the event: Confirmed

Quality testing

Tested: 320 automated tests (309 backend, 11 frontend). Security rules are tests too: localhost-only API, safetensors only, held-out tasks never used for training, no auto-activation, sandbox blocks all non-mock traffic. End-to-end evaluation run in the UI with a scripted test model: 18 episodes on held-out tasks, 33.3% success, 0% errors, 100% safety. Real Qwen3-VL-2B on an RTX 3050 (6 GB). What broke: The real model solved 0 of 21 tasks. At first every action was invalid: it wrote coordinates as lists. A prompt fix cut invalid actions to 51%, but it still repeats clicks. A short QLoRA run halved the training loss (1.14 → 0.53) but did not change behaviour. Biggest flaw: cold start. The loop learns only from the model's own successes, so at 0% there is no data and no v2. The fix is labelled seed demonstrations.

Worth 20 of the 100 points.

Running it

No credentials needed. Open the demo link in any browser. Nothing to install. Models → Register MockLAM → Activate. Settings → Load evolution workflow template. Workspace → Run Evaluation on LAM Core. Watch the live stages, then open Experiments → Open Report. To run locally (WSL2/Linux, Python 3.11+, uv, Node 20+): follow the "Demo" section of README.md, then cd backend && LAM_SANDBOX_DEV=true uv run python -m app serve and open http://127.0.0.1:8000.

Project ownership

The team keeps the rights to this project. Judges were given access for verification only.