GridSignals

GridSignals turns the public record around US energy companies into scored, sourced signal cards — each mapped to one of 16 Microsoft security products across 62 trigger-product pairs — and surfaces them through a nine-page FastAPI + HTMX web UI. The ingestion pipeline fetches from 13 classified or stored sources (SEC EDGAR submissions and full-text search, Federal Register, NERC enforcement dockets, CISA KEV, NVD CVE, ransomware.live, security-press RSS, EIA plant capacity, and more), resolves each item to one of 172 watchlist entities via deterministic CIK/ticker/LEI match with fuzzy fallback, classifies it into a trigger type, and applies a decay-adjusted score across five persisted components. The entire pipeline runs on Python stdlib — no third-party packages — and the only recurring external cost is a budget-capped Claude Haiku 4.5 accuracy judge ($0.016 measured on a 5-signal run; $0.50 ceiling). 337 commits, 115 PRs, 14 versioned migrations, 84 hermetic test modules, in 9 active days.

The build runs config-as-data throughout: 16 products, 21 trigger types, 172 watchlist entities, scoring weights, and two combo rules all live as CSV seeds loaded into SQLite at startup, so the operator can tune the system through the Admin UI without a code deploy. A 1,225-line packaging test (`test_packaging.py`) enforces the stdlib-only constraint at CI time — among other things, it pins Streamlit's absence as a regression guard after the UI was rebuilt mid-build on FastAPI + HTMX + Tailwind. The combo engine needed a composable rule grammar; the stdlib-only constraint ruled out `eval`/`exec` and produced a hand-written `logic_expr` parser instead. Migrations are versioned and checksummed append-only; a mismatch on an already-applied file is a hard error, preventing silent schema drift across the 14-migration history. SIGTERM handling required a two-pass `/proc` walk to relay the signal to every cron-spawned job descendant, because tini's `-g` flag does not reach processes that cron places in their own sessions — a failure mode that only surfaced when stale lock files blocked the next startup.

Key modules

Key features

• All 13 data sources are public-record only — GET/RSS/JSON, no ToS-restricted scraping, no credential bypass. User-agent identifies the tool: GridSignals/0.1 (+https://github.com/Doogit/GridSignals).
• Single-writer ingestion lock serializes all database writes and is released on SIGTERM via a two-pass /proc relay to cron-spawned descendants, preventing stale lock files from blocking the next container startup.
• Incident outreach is gated by evidence tier: confirmed (8-K Item 1.05 or company statement) enables account-level product plays; corroborated (security journalism) enables sector-level plays; unconfirmed early-warning (ransomware tracker) is operator-only, outreach suppressed, card labeled with the absence of confirmation.
• Security Copilot plays are suppressed for known or likely US government cloud tenants.
• PII guard enforced at commit time via .git/hooks/pre-commit and commit-msg hooks scanning staged content and commit messages against a pattern file.
• Every Admin config edit is appended to a config_audit table — an immutable provenance trail of every weight or half-life change, with the config version stamped per scoring run (migration 0011) for reproducibility.

• The SIGTERM relay in deploy/entrypoint.sh required a two-pass /proc walk to signal every cron-spawned job descendant — tini's -g flag does not reach processes that cron places in separate sessions. The function is tested against a real three-level process tree in test_packaging.py:623–727.
• Entity resolution uses a 25-case adversarial fixture set (tests/fixtures/adversarial_cases.csv) covering bare-name collisions ('Dominion', 'Chord', 'Range', 'Constellation'), CIK zero-padding, ticker case-insensitivity, fuzzy typos, suffix-strip near-twins, and an unknown-company case.
• The Streamlit UI was replaced mid-build with FastAPI + HTMX + Tailwind; test_packaging.py pins Streamlit's absence at line ~332 as a packaging regression guard so it cannot quietly reappear as a transitive dependency.

• A stdlib-only constraint is not just a dependency policy — it forces safer code paths. The combo engine needed a composable rule grammar; ruling out eval/exec produced a hand-written logic_expr parser that is auditable, testable, and grammatically bounded by construction.
• Config-as-data beats hard-coded logic for an operator-facing tool: products, triggers, scoring weights, watchlist entities, and combo rules all seed from CSVs into SQLite. The operator tunes the system through the Admin UI without a code deploy, and a new instance comes up fully configured from the seed files alone.
• Honest empty states are documentation, not embarrassment. The account-specific tier measured 0 of 1,886 EDGAR submissions with a material cybersecurity incident (8-K Item 1.05) — not a broken classifier, a structural truth about filing frequency. Publishing the exact count and the structural reason preempts a misfiled bug and explains the gating logic in one sentence.

The stdlib-only pipeline discipline — zero third-party packages for ingestion, classification, scoring, and audit — as the default for public-record tools where supply-chain surface matters. And config-as-data: seeding operator-tunable parameters into SQLite rather than hard-coding them means the next tool ships with an Admin UI and a tunable surface from day one.

Case study in progress.