← All sessions
Talk · 25 min

OCR at Swiss Post: The Case for Small Language Models

Time
14:20–14:50
Room
Studio 4
Times and rooms can change at any time, also during the conference. This page always shows the current schedule, check it on the day.

Everyone reaches for a frontier LLM by default — and then the bill, the latency, and the data-privacy questions arrive. But public routing research (RouteLLM, Microsoft's Hybrid LLM) shows that 40–60% of typical queries can be handled by a smaller model with no measurable drop in quality — and for specialized, bounded tasks, that number easily explodes toward the vast majority.

At Swiss Post, we put this to the test. We built a centrally self-hosted platform around a Small Language Model, served with vLLM on Kubernetes with GPUs, using OCR as our real-world case. The result: lower cost, better latency, full data sovereignty — and far less operational complexity than the "you can't self-host in production" narrative suggests.

This talk makes the case for SLMs as a serious production strategy, not a compromise. We share a candid comparison of cost, accuracy, and performance against LLM alternatives, how we kept sensitive citizen data fully in-house, and the architecture that makes self-hosting genuinely approachable.

The core message: an LLM is not a Swiss Army knife. For a huge share of real workloads, a specialized, self-hosted SLM is cheaper, faster, more private — and easier to ship than you'd expect. We close with the question that reframed our thinking: how many of your requests actually need an LLM in the end?

Main tools & technology
vLLM — the inference engine at the core of the self-hosted SLM platform (served on Kubernetes with GPUs).
Topics
Local AI, Inference & Open ModelsSoftware Architecture with AIEnterprise AI Adoption & Governance
Become a sponsor

Put your tools, products, and engineering work in front of builders working with agentic AI.

Sponsors & Partners
Main Sponsor
copebit — Main Sponsor
Gold
AWS TD SYNNEX Exasol
Silver
Atlassian Flagsmith LogicStar namespace re:cinq
Hands-On
OpenAI — Workshop Partner
End-User
Migros Online — End User Partner
Education
FHNW Hochschule für Informatik — Educational Partner
Bronze
adesso BI Concepts dihai Keera Noser Engineering Peak Scale Puzzle ITC Team Extension Trustsoft Your Sidekicks AG
Barista
BI Concepts, Barista Sponsor hagspihl.com, Barista Sponsor yardr.dev, Barista Sponsor
Community
AI & ML Events All Day AI CH Open dev.events Global AI Zurich Java User Group Switzerland Rocket Engineers SwissAIJob SwissDevJobs ZurichJS Conference