AI matcher: blending a deterministic score with an LLM verdict
ApplyDesk scores 100 job offers in a fraction of a second right in the browser, and only brings out the LLM for the offers that deserve it, to keep the budget sane. It's one of the more useful things I've built for my own side-project.
Triage 100 offers a day, without burning the LLM budget
ApplyDesk scrapes around 100 job offers a day from 5 sources (France Travail, Adzuna, HelloWork, Remotive, WeWorkRemotely). As a user, I want to immediately know which offers deserve a closer look. I don't want to wait 30 seconds per offer for the AI to think, nor pay for an LLM call on every row.
Two constraints: fast (sub-second) and explainable (I have to see why an offer matches). The LLM alone doesn't hit either one, and heuristics alone lack nuance, so the fix is to combine both.
Two layers, two moments
1. Deterministic score (front, synchronous)
Runs in TypeScript in the browser, on every offer, the moment it shows up. 0 to 100 %, transparent and debuggable. This is what powers the '95 %' badges, sorts the list, and feeds the daily digest.
2. LLM verdict (back, async, on demand)
When the user opens an interesting offer, we call Gemini with a structured prompt that returns a ternary verdict (apply / check / skip) plus 3 concrete reasons. Cached in the database: we never pay for the same call twice.
4 axes, opinionated weights
The deterministic score weights four axes:
Skills
Weighted overlap between CV skills (extracted via AI parsing, see ApplyDesk) and the offer text. Skills from the last 2 experiences weigh 1.0, older ones 0.4, categories without usage 0.2. Saturation at 8 matches: beyond that, +1 skill no longer moves the score.
Title
Profile target_roles and headline tokens (minus stop-words: 'Full', 'Stack', 'Senior'…) must appear in the offer title. One match saturates the axis.
Location
Compare preferred_locations with offer.location. No preference → neutral (0.5). Match → 1. No match → 0.
Fit
Contract type (CDI primes, others half-point) and remote when offered. A weak signal, but useful to break ties between '90 % matches'.
Three subtleties that move the quality needle:
Recency bias
A senior who did 8 years of Java then 2 years of Python isn't 'looking for Java': her recent profile is Python. Weighting by recency avoids scoring high an offer that only matches the 5-year-old version of her.
Accent-insensitive normalisation
'développeuse' must match 'developpeuse'. We normalise via NFD + diacritic stripping. Critical on the FR market where offers mix both spellings.
Boundaries on short skills
Without boundaries, 'Go' matches 'Google'. Fix: skills < 5 chars → regex with non-alphanumeric boundaries. Skills ≥ 5 chars → direct substring (javascript matching javascripteur is fine).
The verdict has three states
On the LLM side, we could have asked for a continuous score ("rate this offer 0–100"). Bad idea: LLMs are poor at fine arithmetic and spend their time wavering between 73 and 78. We ask instead for a ternary verdict:
yes clear fit, should apply
maybe fit but with doubt (borderline seniority, partial stack, location to clarify)
no clear mismatch (seniority, main stack, contract type, location)
Plus 1 to 3 concrete reasons mixing positives and negatives. That's what shows under each offer in the daily digest: 'Spring Boot stack is a perfect match', 'But salary not stated', etc.
The prompt is structured (Pydantic schema on the Python side, JSON mode on Gemini): no regex on free text, no fragile parsing. We get the object, save it, render it.
The verdict lives in the database
The verdict is expensive (~3-5 seconds, a fraction of a cent per offer). We persist it directly in the offer table with 3 columns: verdict, verdict_reasons, verdict_generated_at. Three consequences:
- We pay once per offer, never twice.
- The user gets the verdict back after refresh, logout, navigation.
- We can invalidate selectively (verdict_generated_at > 7 days → re-run) without touching any cache layer.
What we keep from it
- Never ask the LLM what a pure function can do. Counting skills, normalising a string, comparing arrays: that's JavaScript, not a $0.0003-per-token API call.
- The LLM shines on nuance, not arithmetic: 'this offer asks for Spring 6 but your CV stops at Spring 3, which is OK if the team mentors'. No heuristic captures that.
- Structured outputs (JSON mode + schema) change reliability. No more regex, no more retries on text that won't parse.
- Persist the verdict in the database, not in memory cache: it survives deploys, crashes, logouts.
- The deterministic score is debuggable: you see why an offer matches by reading the breakdown. The LLM alone is a black box. Combined, you get explanation and nuance.