all Case Studies
/

How Ottimate improved AI for AP automation with careerflow

"Our automation gets better the more it learns from the cases it gets wrong, and Careerflow is the human validation layer in that loop. Their team gives us accurate, reviewed ground truth on exactly the invoices our models struggle with most: the handwritten ones, the low-light scans, the messy layouts. That feedback is what moved our accuracy on complex documents from 60% to 85%. They've become a real extension of our AP automation team."
Krishna Janakiraman
CTO, Ottimate

About

Ottimate builds AI-powered accounts payable automation that catches overpayments, stops fraud, and helps finance teams close their books faster. Its platform digitizes invoices and pulls out the data finance teams depend on.

Automation handles clean, repeatable invoice formats well. But a real share of documents still defeats it: handwritten invoices, low-light and poor-quality scans, irregular vendor-specific layouts. Those are the cases where accurate human-labeled data matters most. It clears the work automation cannot yet do, and it becomes the ground truth that trains and validates the next generation of models.

Careerflow built the human data team behind that work. In under six months, a team that started at five people and grew to fifteen helped move Ottimate's model accuracy on complex invoices from 60% to 85%, while processing more than 50,000 invoices a month.

At a glance: the Ottimate partnership

Ottimate's AP automation handles standard invoices well, but the hard cases (handwritten, low-light, irregular layouts) need human-labeled data to clear them and to train the next generation of models. Careerflow built the team behind that work. Here's what to know.

  • The client:Ottimate, an AI-powered accounts payable automation platform that digitizes invoices for finance teams.
  • The problem:The hardest invoices involve complex calculations, detailed annotations, and multi-step reasoning, and they demand near-perfect accuracy because errors carry real financial impact. Automation alone could not reach that bar, which dragged down model accuracy and called for accurate human labels at scale.
  • The approach:Careerflow runs as the human validation layer inside Ottimate's closed-loop training cycle, labeling and correcting the hardest invoices and feeding that ground truth back into retraining.
  • The team:Scaled from 5 to 15 dedicated contributors, built through a multi-step vetting process, now self-sufficient with internal mentors and QA.
  • The volume:Grew from 41,222 invoices in March to a projected 60,000 in June, with peak-period throughput doubling to nearly 20,000.
  • The result:Model accuracy on complex invoices improved from 60% to 85%, and the engagement is now a long-term partnership expanding into new dataset types.

The problem: the invoices automation can't read

Ottimate's automation already performed well on standard, repeatable invoices. The hard problem was the long tail: handwritten documents, degraded and low-light scans, and irregular layouts. Many of these invoices are genuinely complicated, involving complex calculations, detailed annotations, and multi-step reasoning to read correctly. And the accuracy bar is unforgiving, because a wrong number on an invoice has real financial consequences. These cases drove model accuracy down, and more automation alone could not fix them. They needed large volumes of accurately labeled examples, plus disciplined human review to benchmark what the model produced.

Volume made it harder. Invoice flow was unpredictable, with sharp monthly spikes. Ottimate needed a partner that could do several things at once:

  • Produce high-quality labeled training data at scale, including on the complex edge cases
  • Provide human validation to measure and improve model accuracy
  • Scale capacity up and down as demand shifted, without disrupting output
  • Hold labeling quality steady while volume grew
  • Operate as a long-term extension of the Ottimate team

The work was not simple data entry. Invoices arrive in varied and sometimes unusual layouts, so contributors had to learn Ottimate's process, tools, and quality bar. That made trainability and consistency matter as much as raw speed.

A closed-loop training model

The engagement was built around a closed loop, with Careerflow as the human validation layer inside it. This was not a one-time labeling project. It was a continuous cycle that compounds:

  1. Invoices arrive, including the handwritten, low-light, and irregular cases automation handles least reliably.
  2. Ottimate's models attempt extraction. The low-confidence and complex cases route to the human team.
  3. Careerflow labels and corrects each one, producing accurate, human-verified ground truth.
  4. Careerflow validates model output against that ground truth, showing exactly where and how the model gets things wrong.
  5. The corrected data feeds retraining, so the model improves on the cases it used to miss.
  6. Automation absorbs more volume, the human team focuses on the remaining hard tail, and the loop repeats at a higher baseline.

Each pass raises accuracy and shifts more work to automation, while human validation keeps the model honest on the cases that matter most. This is the mechanism behind the move from 60% to 85% on complex invoices.

Building a team for quality at scale

The goal was never just to clear invoices. It was to build a workforce that produces reliable, model-ready data: accurate labels for training and disciplined review for validation, at a volume and consistency that actually moves model accuracy.

A multi-step vetting process

Careerflow pulled the strongest contributors from a large applicant pool through a multi-step vetting process that combined structured assessments, AI-assisted interviews, telephonic screening, and human evaluation.

Stage Volume
Applications received 600 to 700
Assessment qualified ~450
AI interview review ~100
Telephonic screening 40 to 50
Final team selected 15

Candidates were judged on more than technical aptitude. Communication, attention to detail, learning agility, and problem-solving all counted, because labeling quality on complex documents depends as much on the ability to learn and adapt as on prior knowledge.

Productivity and volume

The team's output roughly tripled as it matured, while quality held steady. Per-contributor productivity grew about 3x from go-live, and the pace stood out to Ottimate. When the team crossed 200 invoices a day, Akshay, Ottimate's trainer, noted:

"We have many people in our current team who are doing 200+ every day, but they have many years of experience compared to this team, which is just a few months old."

Absorbing demand spikes

A core requirement was handling unpredictable volume surges without losing quality or speed. As the team matured, peak-period capacity climbed sharply.

High-volume periode Invoices labeled
Month 1 9,818
Month 2 15,043
Month 3 19,727

Peak throughput rose 100.9% between Mont 1 and Mont 3. In 3rd Month alone, peak periods accounted for nearly 20,000 invoices, about 35.5% of the month's output.

The team also flexes with demand. When Ottimate needs more hands, Careerflow vets, trains, and onboards new contributors in under 24 hours, then scales back down once the spike passes, so Ottimate gets surge support without carrying a permanent surplus. During one unexpected high-volume weekend, Careerflow recruited, onboarded, and prepared ten additional contributors in under a day. Before touching live invoices, each completed training invoices, got mentorship from experienced team members, and passed a quality and readiness review, so the added capacity never came at the cost of data quality.

"The flexibility offered has made the quality of life so much better. I am able to take care of my family while continuing to grow in a quality role."
Shaini Menon
Careerflow contributor

Results

In under six months, Careerflow turned a data requirement into a partnership that measurably improved the product.

Model accuracy. Ottimate's internal accuracy on complex invoices improved from 60% to 85% over the engagement, driven by the team's human-labeled training and validation data. The gains came specifically on the hard cases: handwritten invoices, low-light and degraded scans, and irregular layouts that automation alone could not handle.

Operations. The team scaled from 5 to 15 dedicated contributors while holding quality steady, roughly tripled output, grew monthly volume from 41,222 to a projected 60,000, and doubled peak-demand throughput. It also became self-sustaining, training, mentoring, and running QA internally, which reduced Ottimate's dependence on client-side resources.

Why it worked

This engagement grew beyond staffing into a data partnership that moved a real product metric. Four things drove the outcome:

  1. A data-quality focus, treating every processed invoice as model-ready training data and every review as validation.
  2. A rigorous multi-step vetting process that selected for trainability and consistency, not just experience.
  3. Internal mentorship and QA, with experienced contributors leading training and review.
  4. Elastic, surge-ready capacity, onboarding new contributors in under 24 hours without diluting quality.

What started as a single invoice-processing engagement is now a long-term partnership. Careerflow and Ottimate are planning new work together, including new dataset types and workflows beyond invoice processing.

Build The Eval Data Behind Your Models

Careerflow builds high-quality, human-labeled evaluation data for AI teams. We source and vet domain experts through multiple layers of screening, build the labeling tools each metric needs, iterate with you on definitions and edge cases, and review every set through a multi-opinion QA process. We work as an extension of your team, at your speed.