# Host Prompt Finder

Status: published to the actual Next.js game on 10 October 2026. See the
release record for deployment identities and live verification.

The host can search or browse the original 40-card deck, inspect the exact card,
choose it, and explicitly deal it to the room. The existing random-card button
still works. Current and previously answered cards are excluded from selection.
The existing admin-token authorization and host rate limit protect the deal.
Client-supplied prompt text is never accepted: the server resolves the chosen ID
against `seedCards`.

## Decision and ownership

Saran directed an AI-assisted implementation and review of this enhancement.
The product-specific work is the search/deal workflow, source-preserving ranking,
local model preparation, evaluation and fallbacks. Stanford trained GloVe;
this project does not claim to have trained that model. This is retrieval, not
RAG or generated song advice. LangChain was not required for this small corpus.
It uses no recordings, lyrics, room answers or player transcripts as retrieval
data. The prompts are the existing original game deck.

BM25 keyword search is the default. An optional comparison combines 65% cosine
similarity of averaged, document-frequency-weighted GloVe vectors with 35%
bounded BM25. The first 20,000 alphabetic lowercase pretrained words are
normalized and quantized to integers. Browser calculations use plain TypeScript;
there are no dependencies to install, paid calls, WASM policy changes or Python
servers at runtime. Python only prepares and evaluates the checked-in model.

## Reproduce

1. `py -3.14 scripts/prepare-prompt-model.py` downloads the public 69 MB source
   once to ignored `.data/prompt-model`, verifies the publisher's checksum, and
   generates `public/models/prompt-glove-50.json` (3,315,565 bytes).
2. `npm run prompts:evaluate` runs the actual TypeScript search engine through
   esbuild, then Python's standard library scores the frozen labels. No model
   download is needed for evaluation with the checked-in browser model.
3. `npm test -- tests/unit/prompt-search.test.ts tests/api/prompt-selection.test.ts`
   checks ranking contracts and selected-card validation.
4. `npm run dev:mock -- --hostname 127.0.0.1 --port 3117`, open `/create`, create
   a mock room, and choose **Find a prompt**. Try `road trip`, enable the optional
   semantic comparison, then choose and deal a card. No live providers are used.

## Frozen evaluation, 10 October 2026

An independent agent authored 24 answerable English requests, six unrelated or
nonsense requests, and four separately labelled negation/ambiguity challenges
before seeing scores. These are synthetic relevance judgments, not user studies.
No parameters or labels were tuned after seeing these results.

| Measure | BM25 | Semantic + BM25 |
| --- | --- | --- |
| Expected card in first 3 (24 answerable) | 14/24 | 14/24 |
| Expected card in first 5 (24 answerable) | 14/24 | 15/24 |
| Mean reciprocal rank at 5, the UI limit | 0.5833 | 0.5917 |
| Empty result for unsupported requests | 5/6 | 3/6 |
| Challenge hit at 3, separate set | 3/4 | 3/4 |

There is no top-three improvement. The model finds one additional expected card
within the five shown, but rejects unrelated requests less reliably. That is why
keyword search remains the default and semantic search is labelled experimental.
The report also retains full-ranking MRR as a diagnostic; it includes results
the UI never shows and must not be presented as the user-facing ranking metric.

Examples of actual failures: the semantic method suggests cards for a SQL-join
question, a bread recipe and a Tokyo weather forecast. The threshold of 0.55 is
an uncalibrated similarity gate, not a confidence score or safety guarantee.
Word averaging ignores order, handles negation poorly and has a restricted
English vocabulary. Filters and exclusion of previously answered cards are not
part of the relevance benchmark. They have separate functional tests.

See `evaluation/prompt-evaluation.json` for every label and result, and
`evaluation/prompt-rankings.json` for full scores and SHA-256 fingerprints of the
fixture, corpus, model and runtime. See `public/models/NOTICE.txt` for attribution.

## Boundaries and release

The optional 3.3 MB model loads from the app's own static assets only on click.
Search queries stay in component memory and are cleared when the drawer closes;
the model may remain in HTTP cache. Browser-local search does not request song
recommendations from the internet. A failed or timed-out model download leaves
keyword search and deck browsing usable. A late download cannot override a
subsequent choice of Keyword mode.

Existing Supabase, player privacy, microphone, YouTube and AI provider flows are
unchanged. No secrets or schema migrations were changed. The owner authorized
publication of the reviewed game and portfolio on 10 October 2026. The existing
Supabase project was found paused during live verification and resumed on its
existing Free plan. Mock tests alone do not establish production behavior;
the release record separates those checks from live verification. The reviewed
pre-existing gameplay edits were included in that authorized release.

The old Astro facilitation-planner branch `codex/listening-party-rag` is unrelated
reference work, not this game enhancement or the live domain's source. It has
not been merged or removed.