AI Enterprise SolutionsGPT & Claude

Evalu

Enterprise customer support teams across telecom, fintech, and digital services do not suffer from a lack of customer feedback; they suffer from fragmented channels and operational amnesia. Critical complaints in call-center transcripts, CRM tickets, live chats, and app reviews are trapped in silos, treated as isolated transactional tickets rather than root systemic failures. Consequently, identical issues repeat endlessly, driving up support costs, customer churn, and repeat-contact rates. Furthermore, enterprises cannot safely leverage commercial LLMs on raw customer communications due to the strict risk of leaking personal identifiers (PII). Evalu solves this by bridging the gap between frontline customer pain and corporate department accountability. It deterministically strips sensitive PII before inference, uses an internal AI model to extract structured root causes, groups recurring complaints into explainable priority clusters, assigns accountable department owners (Billing, Network, Support, Product), and continuously tracks whether operational interventions actually eliminate repeat customer contacts.

Open the demo Video GitHub

Preliminary score

65.0/ 100Promisingaverage of 2 judges

Value for the user19.0 / 25
A specific problem and a clear outcome from solving it.
Prototype and use of AI20.0 / 30
A working core scenario, and what AI actually contributes to the result.
Quality testing12.0 / 20
Tests, examples of failures, and comparison with the current approach.
Feasibility8.5 / 15
Data requirements, running costs and a clear next step.
Originality5.5 / 10
A meaningful difference from common solutions.

What the judges wrote

GPT-6 AstraPreliminary70/100
Strengths
Your workflow connects fragmented support feedback to prioritized issues, accountable department actions, and measurable repeat-contact outcomes. Your explicit AI-versus-rules provenance, schema validation, and fallback handling make operational limitations visible rather than hiding them. Your documented 21-test suite, reproducible setup, and clear synthetic-data disclaimers strengthen the submission.
To improve
Provide an end-to-end live-model trace and measure extraction accuracy on held-out Azerbaijani, Russian, and English examples against rules-only and manual triage baselines. Scope your privacy guarantee to tested identifier patterns: regex masking does not establish zero leakage for names, addresses, or unfamiliar identifiers, and retained raw text needs protection. Validate outcome tracking in a customer pilot, since the seeded reductions demonstrate presentation rather than intervention effectiveness. Estimate inference and hosting costs per 1,000 interactions and define a pilot with secure data access and an explicit adoption threshold.
Claude Opus 5.5Preliminary60/100
Strengths
Your pipeline is concretely engineered and runs end to end: CSV ingestion, deterministic PII redaction tuned for Azerbaijani, Russian and English formats, schema-validated structured extraction with a retry, clustering, owner assignment and persisted actions. Redacting before inference is a real enterprise blocker you address directly, and labelling every result as AI or rules-based fallback is honest design. Your three failure modes (invalid JSON, API outage, ambiguous multilingual input) are explicit, testable and visible in the UI. The explainable 4-factor priority formula makes the ranking auditable for department owners.
To improve
Your 21 tests check that the software behaves correctly, but they do not measure how accurate the AI is. Hand-label your 45 interactions and report topic, owner and urgency accuracy for gpt-4o-mini against the rules-based fallback. That comparison would also show whether the model is essential, since the demo runs fully without an API key. Your closed-loop result (88.5% drop) is seeded synthetic data, so present it as an illustration rather than evidence. Add per-interaction inference cost, a baseline for how support teams triage recurring issues today, and the time it saves, plus a concrete pilot step with a real support team.

The team

Aytac Maharramova
designer
Nuray Shiraliyeva
business
Kanan Gasimzada
data
Kamal PashayevLead
developer

Disclosure

Required by the rules. Judges verify it against the repository.

Models: Evalu utilizes a modular, environment-configured AI Provider Adapter (AI_BASE_URL, AI_API_KEY, AI_MODEL, defaulting to gpt-4o-mini / OpenAI-compatible endpoints) strictly isolated behind a deterministic privacy boundary: Role of AI: Ingests pre-redacted text to generate strict structured JSON containing sentiment (positive | neutral | negative), canonical topic, single-sentence summary, urgency (low | medium | high), suggested_owner (Billing | Network | Support | Product | Other), repeat_contact_risk (low | medium | high), and confidence score (0 to 1). Validation & Retry Layer: Every model output is schema-validated via validateAnalysisSchema(). Invalid responses trigger an immediate retry with specific structural error feedback in the prompt. Deterministic Multilingual Fallback Engine: If API credentials are absent, endpoints timeout, or validation retries fail, Evalu transparently engages a zero-dependency rules-based taxonomy supporting Azerbaijani, Russian, and English telecom domains. Provenance & Ethics: Inferences are explicitly labeled as either AI (model_name) or rules-based fallback with full fallback rationale. Fallback results are never misrepresented as AI inferences.
Data: Evalu operates on a fully synthetic, multilingual benchmark dataset (data/synthetic_interactions.csv) consisting of 45 customer interactions balanced across Azerbaijani, Russian, and English over Call Centre, Chat, CRM Ticket, and App Review channels: Realistic Telecom Scenarios: Models five operational categories—unexpected data charges, mobile network degradation, SIM/OTP authentication delivery failures, mobile app crashes, and positive support feedback. Synthetic PII: Realistic synthetic phone numbers, email addresses, 16-digit credit/debit card numbers, account codes (ACC-xxxx, CUST-xxxx), and national FIN codes are embedded in the dataset and deterministically scrubbed prior to model inference. Zero real customer data or production telemetry is utilized. Closed-Loop Benchmark: Features a seeded synthetic before/after dataset tracking the impact of a Billing intervention on "Unexpected Data Charges" over a 7-week trajectory (W-2 to W+4), demonstrating an 88.5% drop in weekly mentions and an 83.4% relative reduction in repeat contacts. Marked with a prominent synthetic demo disclaimer in the application interface.
Components: Deterministic PII Redaction Engine (server/pii.js): Regex-based pre-inference sanitizer removing phones, emails, card numbers, and account IDs before external transmission. AI Provider Adapter & Schema Validator (server/ai.js): Environment-driven adapter with JSON schema enforcement, single-retry error recovery, and multilingual rules-based fallback. Clustering & Explainable Priority Engine (server/cluster.js): Topic aggregation engine computing volume, negative rate, affected channels, sample redacted evidence, and ranking clusters via an explainable 4-factor formula: Priority = (0.30 × VolumeNorm + 0.25 × NegativeRate + 0.25 × UrgencyWeight + 0.20 × RepeatRiskWeight) × 100. Lightweight Server Persistence Store (server/store.js): Server-side JSON store (server/data/db.json) persisting ingested interactions, operator actions, and closed-loop benchmarks across reloads. REST API Server (server/index.js): Node.js Express server exposing endpoints for CSV ingestion, interactions, clusters, actions, closed-loop telemetry, system health, and failure mode simulations. Evalu Interactive Operations UI (src/pages/EvaluPipelinePage.jsx): React 19 / Vite workspace featuring live PII redaction comparison, structured model JSON cards, priority contribution breakdown bars, interactive action creation, closed-loop impact trajectory, and failure simulation triggers. Automated Verification & Demo Suite (tests/, scripts/demo_flow.js): 21 passing test cases and an executable end-to-end demo script runnable via npm test and npm run demo
Built during the event: Confirmed

Quality testing

Evalu enforces strict end-to-end quality assurance across 21 automated unit and integration tests executed via Node.js native test runner (node --test), verifying: Deterministic PII Redaction: Complete pre-inference masking of international/local phone numbers (+994, +7, +1), email addresses, 16-digit card patterns, and patterned account/FIN codes across Azerbaijani, Russian, and English text. Strict JSON Schema Validation & 1-Retry Policy: Deterministic validation of model outputs against required enums and types, with an automated single-retry loop before fallback engagement. Explainable Priority Scoring: Mathematical verification of the 4-factor ranking formula and factor contribution calculations. Resilience & Documented Failure Modes: Explicit test coverage and user-facing attribution for three documented failure scenarios: Invalid Model JSON: Validation failure triggers retry, then safely engages the rules-based fallback with explicit provenance tagging (model_used: "rules-based"). Model/API Unavailable: Network timeouts and HTTP 503 errors trigger an immediate, graceful fallback without crashing. Ambiguous Multilingual Feedback: Low-semantic or mixed prompts are tagged with low confidence (0.45), categorized as "General Inquiries / Ambiguous", and routed to human review. End-to-End Integration Pipeline: Full verification covering CSV ingestion -> PII scrub -> model extraction -> cluster ranking -> action assignment -> server persistence -> closed-loop outcome verification. Code Standards: Clean compilation with 0 ESLint errors and production bundle verification via Vite.

Worth 20 of the 100 points.

Project ownership

The team keeps the rights to this project. Judges were given access for verification only.