Evaluation matched to the decision
Offline and online evaluation, via automated scores validated against human judgment, with metrics motivated by the product decisions to be made.
Multilingual evaluation
Find out what your system does before your users do.
We help teams evaluate speech and language products with evidence they can act on: what works, where it fails, and which failures matter most.
How we can help
Usually one of four things: an evaluation designed around the decision you face, data development to support it, an outside read on the evidence once it exists, or a written account of why the system fails.
Multilingual data development
Data development to support an evaluation design: acquisition, synthesis, annotation, and quality control.
An outside review of your system
Once an initial evaluation has run, we review the datasets, metrics, and reported results, looking for coverage gaps, leakage, PII risks, and conclusions the data cannot support.
Failure analysis that finds the patterns
Working from the evaluation and audit results, error taxonomies and slice analysis replace vague generalizations with actionable insights.
Selected evidence
A few systems we have helped build, evaluate, and ship to the public.
Eldercare chat assistant
Built the beta of Kedara Guide and instrumented its RAG + dialogue pipeline with evaluation hooks and deidentified OpenTelemetry traces for observability and iterative improvement.
Full stack · RAG evaluation · observability
Meeting summaries for millions
Collaborated on automated chapters, notes, and action items from development through adoption by millions of Webex users, then led extension of the feature to more than a dozen languages.
Millions of users · 12+ languages
Multilingual voice assistant
Led data development, evaluation processes, and product metrics for a meeting room device voice interface, while we expanded query parsing and response generation from English to six additional languages.
NLU + NLG · product metrics · 7 languages
Bring us the system that worries you.
Send us a short note about the product, the decision, and the timeline. We will tell you frankly whether we are the right team for it.