CONTEXT
The work started here.
A real job search needed the machinery of a sales pipeline (daily postings from many sources, scoring, contact discovery, outreach, follow-ups) without automating the one step that has to stay a human decision: sending.
BUILD
What was built.
A local-first, single-user job-search pipeline. A dozen-plus source adapters pull postings from ATS (applicant tracking system) APIs and job aggregators every day; deterministic hard filters drop the obvious misses; a cheap LLM triage pass scores every survivor; a structured deep-fit pass scores the top N; and a weighted 0–100 priority score ranks the queue. The design then finds the recruiter or hiring manager, drafts direct outreach, and waits: the user approves and sends from Gmail, and follow-ups and replies are tracked on a dashboard (the spec; I haven't verified every outreach feature against code). A pnpm monorepo in strict TypeScript (Express API, React 18 + Vite dashboard, Drizzle over SQLite with a Postgres-compatible schema) set up to run as a Windows scheduled task on localhost. About 117k lines of TypeScript including tests, and 3,400+ test cases by static count. 1,352 commits across 15 active days, local-only with no remote.
PROJECT DESIGN
How it was shaped.
Spec and review came before code. The first three commits were the PRD, the decision log, and a review pass that applied 32 findings across security, outreach safety, dedupe, sequencing, and budget. From there the build ran as plan units: one branch per unit in its own git worktree, a separate-context code review on each unit diff, then a local --no-ff merge (232 merge commits). Every architecture, model, and process call went into a numbered decision log, 68 entries by Oct 29. Agents work under a written autonomy contract: keep moving without asking, except for sending outreach, new paid services, writes to the live database beyond an approved script, irreversible schema shapes, or changes to frozen eval labels.
Key modules
Source adapters
A dozen-plus adapters behind one fetch(sinceTs) → RawPosting[] interface. One adapter failing records its error against that source and never fails the run; adapters are tested against recorded fixtures, never the live network.
Dedupe + closed detection
Canonical jobs keyed on normalized company and title within 30 days, with a fuzzy title match at a 0.9 token-sort ratio. Every fuzzy merge writes an event so a bad merge can be split, and a posting is marked closed only after three successful full-listing runs without it.
Two-stage scoring
Hard filters, then triage on every survivor, then deep fit on the top N with structured output, then a weighted priority score computed in two passes. An ensemble path can run k deep-fit calls and aggregate; at k=1 it returns the single run unchanged.
LLM provider layer
Every model call goes through one LLMProvider interface, is validated against a zod schema (a typed runtime contract for the output), and writes an llm_calls row with model, tokens, cached tokens, and cost. A validation failure gets one retry with the error appended, then fails the item, never the run.
Sender + guardrails
The sender loop sends only approved messages. A pure guardrail function decides send, wait, or cancel, checking cancel rules before wait rules, with the clock and timezone injected so it tests without either. A circuit breaker can pause the loop.
Eval harness
A workspace package that runs the triage and deep-fit scorers against a labeled dataset and reports precision, recall, and Spearman rank correlation against fixed gates. Changing a scoring prompt means re-running it.
Key features
Approval as an invariant
The send path is built so a human click is the only way in. An outbound email needs a messages row with status approved, created by a user click on that message, and no config flag skips Approve. Daily caps, warm-up caps, minimum spacing, quiet hours, and do-not-contact are specified as server-side checks in the sender loop, not in the dashboard. The decision itself is a pure function that returns send, wait, or cancel, and it checks cancel rules before wait rules, so a reason to cancel always wins over a reason to wait.
Evals with an independence guard
Triage has to clear precision ≥0.85 and recall ≥0.80, and deep fit a Spearman rank correlation ≥0.6 against the labeled ranking. The labels behind those numbers are written by an agent in a separate context that never sees the scoring prompts, the stored scores, or scorer output; it works from a written rubric, and labels are committed before prompt work starts. Changing a frozen label needs a logged reason. Each label also records where it came from (an accepted draft, Kevin, or an agent), and the labeling helper hides every score while you label.
SECURITY & OPS
The operating decisions.
- A localhost API with an Approve button is still a web target. A host-header allowlist (localhost, 127.0.0.1, [::1]) runs ahead of every route and static file to block DNS rebinding (a trick that points a hostile domain at your own machine), and a CSRF guard requires a custom header and rejects a mismatched Origin on every non-GET route, so a drive-by page can't reach the endpoints.
- Secrets live only in .env or Windows Credential Manager. They're entered from Settings through a write-only route, never stored in SQLite, and the settings API returns an allowlist plus configured-yes/no flags, never a value.
- Model inputs are minimized: the resume's contact block is stripped before any LLM call, contacts reach drafting as first name, title, and company only, and request payloads aren't persisted outside an explicit dev flag.
- Posting text is treated as untrusted input. Prompts fence it off as reference data with delimiter neutralization (so a posting can't close the fence and inject instructions), every output must pass zod validation, and third-party text renders as escaped text, never raw HTML.
- No LinkedIn automation: no cookies, session tokens, login flows, scrapers, or browser automation against linkedin.com. LinkedIn URLs are stored and opened by the user only.
BUILDER NOTES
What the build taught.
- 599 of the 1,352 commits are docs: the plan, decision, and handoff trail behind the plan-unit workflow: unit specs, per-unit notes, decision rows, and dated session handoffs. Docs outnumber feature commits more than three to one (599 to 176).
- Performance is test-enforced. Every GET route registers a latency budget, and tests guard against whole-table reads in request handlers, so a whole-table scan fails the suite before it can slow the dashboard.
- Prompts are versioned like code. Twenty prompt modules each export a version string that is stored on every score and draft row, so any ranking can be traced to the exact prompt that produced it.
LESSONS
What changed afterward.
- Model choice is an eval result, not a cost preference. Triage on the cheaper model missed the recall gate, so the model moved up a tier; the gate made the call, not the bill.
- An eval is only as honest as its labeler's blindness. If the agent writing labels can see the prompts or the scores, it grades toward them; a separate context and a rubric committed first keep the answer key independent.
- Write the dangerous decision as a pure function. Send, wait, or cancel lives in one function with time and timezone passed in, so the most consequential rule in the system is also the easiest one to test.
WHAT CHANGED
The cheaper model failed its own eval.
Triage, the cheap first-pass relevance check, first ran on a cheaper Haiku-class model. It missed the eval's recall gate twice, at low and then medium effort, so triage moved up to Sonnet at medium effort. The model choice is recorded next to the eval result that forced it.
CONTROL
The constraint that held.
No config flag bypasses Approve: an outbound email needs an approved row created by a user click on that exact message, and the sender loop is designed to enforce daily caps, spacing, quiet hours, and do-not-contact server-side.
EVIDENCE
Where to inspect it next.
- A localhost API with an Approve button is still a web target. A host-header allowlist (localhost, 127.0.0.1, [::1]) runs ahead of every route and static file to block DNS rebinding (a trick that points a hostile domain at your own machine), and a CSRF guard requires a custom header and rejects a mismatched Origin on every non-GET route, so a drive-by page can't reach the endpoints.
- Secrets live only in .env or Windows Credential Manager. They're entered from Settings through a write-only route, never stored in SQLite, and the settings API returns an allowlist plus configured-yes/no flags, never a value.
- Model inputs are minimized: the resume's contact block is stripped before any LLM call, contacts reach drafting as first name, title, and company only, and request payloads aren't persisted outside an explicit dev flag.
- Posting text is treated as untrusted input. Prompts fence it off as reference data with delimiter neutralization (so a posting can't close the fence and inject instructions), every output must pass zod validation, and third-party text renders as escaped text, never raw HTML.
- No LinkedIn automation: no cookies, session tokens, login flows, scrapers, or browser automation against linkedin.com. LinkedIn URLs are stored and opened by the user only.
CARRIED FORWARD
What survived the project.
The approval invariant (a human click stored as the row the sender requires, not a setting) and evals whose labeler never sees what it grades.
WRITING
Field notes from this work.
No public field notes for this project yet.