detect_hallucination

Greenfield Prototyping

Move fast without pretending assumptions are facts

When to use

  • "Vibe code" prototyping where requirements are incomplete
  • You want to move fast but still know what's real vs invented
  • Early-stage design where decisions aren't fully specified

The pattern

  1. Collect evidence. Gather spans for requirements, constraints, repo context, experiments. If you can't collect automatically, ask the user.
  2. Extract Facts. Only include requirements/constraints you can point to. Every sentence must end with citations.
  3. Label Decisions. Design choices you're making. Cite evidence if it motivates the decision.
  4. Declare Assumptions. Throughput goals, tech choices not in evidence, etc. These are allowed—just don't smuggle them in as Facts.
  5. Verify Facts only. Run detect_hallucination on the Facts section. If any Fact is flagged, move it to Assumptions or request evidence.

Evidence pack

Suggested spans:

  • S0: Product requirement paragraph / ticket
  • S1: Explicit constraints (no broker, single service, etc.)
  • S2: Privacy/compliance requirements
  • S3: Repo conventions (framework, DB choice, deployment model)
  • S4: Experiment results (throughput, latency, benchmarks)

Output format

Facts (must be cited)

  • The API must expose POST /events and return 202 quickly. [S0]
  • Events are stored in Postgres. [S0]
  • v1 must not use a message broker. [S1]

Decisions (design choices)

  • Use an events_raw table and a background worker. [S0][S1]
  • Use request validation on the event payload. (Motivated by "store events" but not required.)

Assumptions (declared)

  • Throughput target: 1k req/s. (Assumption)
  • Exactly-once semantics. (Assumption)

Verifier settings

detect_hallucination(
  answer="Facts: The API must expose POST /events...",
  spans=[...],
  require_citations=true,
  context_mode="cited",
  default_target=0.9  // prototype-friendly strictness
)

Copy/paste prompt

Produce three sections: **Facts**, **Decisions**, **Assumptions**.
Every Fact must be cited [S#].
Run detect_hallucination(require_citations=true, context_mode="cited") on the Facts section only.
If any Fact is flagged, move it to Assumptions.

What good looks like

  • You can move fast—Decisions and Assumptions are allowed
  • Facts are verified and trustworthy
  • Nothing is smuggled in as a "fact" when it's actually an assumption
  • The prototype is honest about what's real vs invented