How the Training Loop Works
1) USER UPLOADS INVOICE
↓
2) PdfPig extracts text (or vision LLM for image)
↓
3) mxbai-embed-large embeds the text → 1024-d vector
↓
4) Cosine similarity finds TOP-3 most similar past training examples
↓
5) qwen2.5:7b gets prompt = system + 3 examples + invoice text
↓
6) qwen2.5:7b returns JSON (constrained by Ollama's format="json")
↓
7) USER REVIEWS, CORRECTS WHERE NEEDED, then either:
• "Save Corrections" — fixes the scan but doesn't train
• "Promote to Training" — saves as a new example for future scans
The model itself is NEVER fine-tuned. We only grow the example library
— which keeps everything 100% local (no GPU training), fast (a few seconds
per scan), and reversible (you can always delete bad examples).
Training Curves
As you add more confirmed examples:
- Extraction accuracy goes up — the LLM sees how YOUR vendors format their invoices
- Manual corrections drop — fields the model used to miss are now in its context
- Confidence scores rise — the model is more sure of its output
- Processing time stays roughly constant — only the top-3 examples are used regardless
Best Practices
- Start with 5–10 examples per doc type. Add more as you encounter unusual layouts.
- Promote examples that have distinct vendor layouts — repeating the same vendor adds little.
- If a vendor changes format, add the new layout as an example — old one stays valid too.
- Edit prompt templates to add company-specific rules (e.g., "TDS column must map to taxAmount").