Rasoul Kaljahi

I help teams find out whether their LLM-based systems work, and where they don’t

I’m Ras Kaljahi, an independent consultant. I design the tests, measures and review processes that tell you how good your AI output is, in English and in other languages.

Tell me about your project

What I do

Evaluating LLM-based systems

Test sets, scoring methods and automatic judges you can trust, so you can tell whether a change to your prompt, model or pipeline made things better.

Multilingual AI quality

Checking how your AI product performs in each language you support, and comparing translation engines and LLMs for your content.

Quality estimation and review cost

Automatic quality checks that decide which outputs need a human reviewer, so you review less without losing track of errors.

I can also build the tooling for any of these: Python services, pipelines and simple dashboards that your team keeps and runs.

Who I work with

Background

I have a PhD in computing, focused on machine translation and quality estimation, and over 15 peer-reviewed publications.

I have written software for about 28 years. Most recently I was a principal software engineer at Oracle, where I worked from 2018. I evaluated and benchmarked machine translation engines and LLMs across more than 35 languages, and built an LLM-based quality estimation system that went into production and cut costs by 50%.

Before that, I co-founded small software companies, so I am used to working directly with business owners as well as engineers.

Working together

Send me a short description of what you are building and what you are unsure about. I will tell you whether I can help, and what the work would look like.

hello@rkaljahi.com