REPOSITORY ANALYSIS

deep-searcher

Deep Searcher includes a frozen-query evaluation of Jev as a search-stopping judge; this experiment is separate from the full research agent.

E2 · source reviewedPublic material or source evidence was reviewed. A fixed-version source review requires checking a pinned revision; this is not a reproduction, performance endorsement, or safety guarantee.
Stars
8,282
Forks
803
License
Apache-2.0
Last commit
2026-09-22

What Jev does here

Jev judges enough_information from the main query and accumulated intermediate context; code compares stopping decisions under a bounded round protocol.

Makes stopping behavior and call traces inspectable; the protocol explicitly does not score final-answer quality.

Fixed-version documentation and implementation material were inspected on 2026-10-24. No 100-query comparison or live API was run. The evaluation directory does not prove the main application enables this path by default.

What task does this project fit?

Evaluate when a research search loop has enough information to stop.

Material analysed:

From input to action

  1. The experiment freezes a 100-query manifest and accumulates intermediate search context.
  2. Jev judges enough_information on the same main question and accumulated context used for the comparison.
  3. Code records round-level decisions, cached calls and durable checkpoints up to the round cap.

Implementation details that matter

Stopping agreement is not answer quality

The manifest protocol explicitly excludes final-answer quality scoring. Matching another model’s stopping time cannot establish that the resulting answer is correct or complete.

Inspect the supporting source ↗

Durable execution matters for interpretable results

full100_core.py hashes URL/body, stores call observations and failed responses, and separates request-rate gates. Cached responses must not be mistaken for newly measured latency.

Inspect the supporting source ↗

When to choose it

Choose this as a stopping-policy experiment, not a drop-in replacement for Deep Searcher’s entire research agent. Measure downstream answer quality before deploying a new stopping threshold.

How to inspect it yourself

Inspect make_manifest in run_full100.py and DurableAPI.request in full100_core.py. Confirm the frozen question set, seven-round cap and cache markers, then distinguish a decision trace from a scored final answer.

Limits of this review

  • No 100-query comparison or live API was run.
  • The evaluation directory does not prove the main application enables this path by default.
Primary material used in this analysis