Why Can AI Read an Entire Document and Still Misunderstand It?
Imagine uploading an invoice to an AI system. It reads the supplier’s name, purchase date, tax amount, and total without a single error. The result looks excellent—until you discover that the system has placed the tax amount in the “amount due” field. It did not misread the numbers. It misunderstood where they belonged. This is one of the problems that character recognition accuracy alone cannot reveal.
Executive Summary
- Accurate character recognition does not guarantee document understanding: costly errors often arise in the relationship between a value and its label.
- Tables with multiple columns and merged cells test structural understanding more than ordinary paragraphs do.
- Structured output, whether Markdown or named fields, preserves information placement and enables automated validation.
- A rigorous evaluation measures text accuracy, structure, and relationships between fields together.
- A new generation of models, including LightOnOCR-3 and Baseer, combines text recognition and structure extraction in a single step.
Reading Characters Is One Thing. Understanding Where They Belong Is Another.
Optical Character Recognition (OCR) answers the question: What was written? Understanding document structure answers a different set of questions: Where was it written? Which heading does it belong to? And in what order should it be read?
Without a label, 225.00 is an isolated number. Under “Tax,” it is correct information. Under “Amount Due,” it becomes a financial error.
The industry has begun measuring this distinction. In the LightOnOCR-3 release dated October 8, 2026, the team used ParseBench, which evaluates content fidelity, tables, charts, semantic formatting, and visual grounding separately. The four-billion-parameter model scored 89.9 for content fidelity, compared with 66.1 for charts and 68.3 for visual grounding. These categories differ, so their scores should not be compared directly. Still, the broader pattern is clear: reproducing words is easier than reconstructing their relationships.
The model also introduced a mode that outputs a labeled bounding box with coordinates for each region on the page, making the output better suited to chunking in Retrieval-Augmented Generation (RAG) systems.
Why Do Complex Tables Confuse Extraction Systems?
In a simple table, each cell follows its column header. But when columns multiply and cells merge, the system may struggle to answer a basic question: Which header does this value fall under?
A merged cell spanning two columns may be assigned to only one. A single item wrapping onto two lines may be read as two separate items. Arabic adds another layer: the table reads from right to left, while numbers run from left to right within the same text, complicating the reading order.
LightOn itself acknowledges that no model leads in every category. In its evaluation of French documents, the four-billion-parameter model achieved the highest overall score of 74.1, while other models performed better on charts and certain long tables. Test on your own document types.
One Invoice, Two Different Outputs
The accompanying image shows a sample invoice created for illustration, followed by two illustrative extraction outputs. Both contain the same numbers. The difference lies in which labels those numbers belong to.
Output One: Text Stripped of Its Structure
The footer’s reading order has been disrupted, separating two numbers from their labels:
Subtotal before tax 1,500.00
VAT 15%
Amount due225.00 1,725.00
Output Two: A Structured Version That Preserves Placement
|
Description |
Amount (SAR) |
|---|---|
|
Subtotal before tax |
1,500.00 |
|
VAT 15% |
225.00 |
|
Amount due |
1,725.00 |
In the first output, the downstream system has to guess. It takes the first number following “Amount due”: 225.00. The error passes unnoticed because every number is genuinely present in the document.
In the second, each value sits beside its label, making automated validation possible:
1,500 + 225 = 1,725.
Why Does Markdown Matter More Than Raw Text?
Markdown is an intermediate format that preserves headings, lists, and tables in plain text that people can read and systems can process. From it, we can extract structured data—named fields and values—using explicit rules instead of guesswork.
This is why the Misraj team has built Baseer around converting Arabic documents into Markdown, while LightOnOCR-3 produces text tagged with its corresponding page regions.
There are three benefits.
First, better chunking of long documents: a section heading stays with its paragraphs, allowing a RAG system to retrieve meaningful context instead of fragments.
Second, automated validation: rules can be applied to fields with explicit names.
Third, traceability: each value can be traced back to its location on the page, allowing a person to review it when in doubt without rereading the entire document.
How Do We Evaluate Extraction Without Misleading Ourselves?
Traditional evaluation measures the distance between extracted text and a reference, using metrics such as Word Error Rate (WER). This is necessary, but it does not reveal that 225.00 has landed in the wrong field when all the numbers are still present.
Three layers are needed together:
- Text accuracy: Word and character error rates.
- Table structure: Tree-Edit-Distance-based Similarity (TEDS) compares the structure of an extracted table with its reference. The Misraj team reported a WER of 0.25 for Baseer on Misraj-DocOCR, alongside TEDS and MARS scores on KITAB-Reviewed. Rigorous evaluation therefore considers both text and structure.
- Relationships between fields: Arithmetic and logical rules, such as checking that the subtotal plus tax equals the amount due. This rule alone would have exposed the invoice error in seconds.
Be aware of pitfalls in the metrics themselves. According to LightOn, distance-based metrics can penalize representations that readers consider equivalent, such as a footnote expressed in HTML rather than LaTeX. Even a simple automated reformatting step can change the score without changing the extracted information. Include tests that accept equivalent representations.
Where Does Misraj Fit? Baseer OCR and Baseer Extract
Baseer is a Vision-Language Model (VLM) fine-tuned by the Misraj team for Arabic documents. It converts them into Markdown rather than producing only flattened text.
According to Misraj’s Hugging Face page, Baseer achieved a WER of 0.25 on Misraj-DocOCR, a benchmark comprising 400 expert-reviewed document images, outperforming Azure AI and Gemini 2.5 Pro. Its value here lies in preserving structure in the output, which every downstream process depends on.
Baseer Extract, meanwhile, focuses on extracting the required data from financial documents: the fields an accounting system needs, rather than everything written on the page.
The two capabilities naturally complement each other. Extracting fields from a document with preserved structure is more straightforward than extracting them from flattened text, while the validation rules described earlier safeguard the transition between the two.
What Should Technology Leaders Do Now?
- Classify documents by the categories that matter to your organization: invoices, contracts, statements, and tables.
- Build a sample of real documents with corrected reference outputs, including complex tables, poor-quality scans, and both Western Arabic numerals (0–9) and Arabic-Indic numerals (٠–٩).
- Require structured output from any system, whether Markdown or named fields, rather than raw text.
- Add arithmetic and logical validation rules before the data reaches the financial system.
- Measure text accuracy, structure, and field relationships separately, and repeat the evaluation whenever the model changes.
The question that should precede every decision is not “How many characters did the system read correctly?” but “How many values ended up in the right place?”
References


