AI matcher: blending a deterministic score with an LLM verdict
How ApplyDesk scores 100 job offers in a fraction of a second in the browser, and keeps the LLM budget reasonable to deliver a reasoned verdict only on those that deserve it. Lessons from a personal side-project.
Triage 100 offers a day, without burning the LLM budget
ApplyDesk scrapes around 100 job offers a day from 5 sources (France Travail, Adzuna, HelloWork, Remotive, WeWorkRemotely). As a user, I want to immediately know which offers deserve a closer look — and I don't want to wait 30 seconds per offer for the AI to think, nor pay for an LLM call on every row.
Two constraints: fast (sub-second) and explainable (I have to see why an offer matches). The LLM alone is neither. Heuristics alone lack nuance. The answer: combine both.
Two layers, two moments
1 — Deterministic score (front, synchronous)
Runs in TypeScript in the browser, on every offer, the moment it shows up. 0 to 100 %, transparent and debuggable. This is what powers the '95 %' badges, sorts the list, and feeds the daily digest.
2 — LLM verdict (back, async, on demand)
When the user opens an interesting offer, we call Gemini with a structured prompt that returns a ternary verdict (apply / check / skip) plus 3 concrete reasons. Cached in the database: we never pay for the same call twice.
4 axes, opinionated weights
The deterministic score weights four axes:
Skills
Weighted overlap between CV skills (extracted via AI parsing, see ApplyDesk) and the offer text. Skills from the last 2 experiences weigh 1.0, older ones 0.4, categories without usage 0.2. Saturation at 8 matches: beyond that, +1 skill no longer moves the score.
Title
Profile target_roles and headline tokens (minus stop-words: 'Full', 'Stack', 'Senior'…) must appear in the offer title. One match saturates the axis.
Location
Compare preferred_locations with offer.location. No preference → neutral (0.5). Match → 1. No match → 0.
Fit
Contract type (CDI primes, others half-point) and remote when offered. A weak signal, but useful to break ties between '90 % matches'.
Three subtleties that move the quality needle:
Recency bias
A senior who did 8 years of Java then 2 years of Python isn't 'looking for Java' — her recent profile is Python. Weighting by recency avoids scoring high an offer that only matches the 5-year-old version of her.
Accent-insensitive normalisation
'développeuse' must match 'developpeuse'. We normalise via NFD + diacritic stripping. Critical on the FR market where offers mix both spellings.
Boundaries on short skills
Without boundaries, 'Go' matches 'Google'. Fix: skills < 5 chars → regex with non-alphanumeric boundaries. Skills ≥ 5 chars → direct substring (javascript matching javascripteur is fine).
Three states, not two
On the LLM side, we could have asked for a continuous score ("rate this offer 0–100"). Bad idea: LLMs are poor at fine arithmetic and spend their time wavering between 73 and 78. We ask instead for a ternary verdict:
yes clear fit, should apply
maybe fit but with doubt (borderline seniority, partial stack, location to clarify)
no clear mismatch (seniority, main stack, contract type, location)
Plus 1 to 3 concrete reasons mixing positives and negatives. That's what shows under each offer in the daily digest: 'Spring Boot stack is a perfect match', 'But salary not stated', etc.
The prompt is structured (Pydantic schema on the Python side, JSON mode on Gemini): no regex on free text, no fragile parsing. We get the object, save it, render it.
The verdict lives in the database — not in a memory cache
The verdict is expensive (~3-5 seconds, a fraction of a cent per offer). We persist it directly in the offer table with 3 columns: verdict, verdict_reasons, verdict_generated_at. Three consequences:
- We pay once per offer, never twice.
- The user gets the verdict back after refresh, logout, navigation.
- We can invalidate selectively (verdict_generated_at > 7 days → re-run) without touching any cache layer.
What we keep from it
- Never ask the LLM what a pure function can do. Counting skills, normalising a string, comparing arrays — that's JavaScript, not a $0.0003-per-token API call.
- The LLM shines on nuance, not arithmetic: 'this offer asks for Spring 6 but your CV stops at Spring 3, which is OK if the team mentors'. No heuristic captures that.
- Structured outputs (JSON mode + schema) change reliability. No more regex, no more retries on text that won't parse.
- Persist the verdict in the database, not in memory cache — it survives deploys, crashes, logouts.
- The deterministic score is debuggable: you see why an offer matches by reading the breakdown. The LLM alone is a black box — combined, you get explanation and nuance.