Why We Built PaperTrail: Moving from AI Paper Summaries to Continuous Scientific Intelligence

Every few months, a new "AI research tool" hits the front page of Hacker News or Product Hunt. Almost without exception, they all do the same thing:

Here's the fundamental problem: Science is not a collection of static text summaries. Science is an evolving, adversarial debate.

A paper published in 2023 demonstrating a 4x inference speedup for 4-bit quantized language models on Apple Silicon is not an absolute truth. It is a finding contingent on specific hardware, quantization algorithms, precision levels, and thermal conditions. When a 2024 paper reports severe degradation under continuous workloads on Qualcomm Snapdragon chips, they aren't necessarily contradicting each other—they're operating in completely different experimental contexts.

And when a seminal paper gets retracted or contradicted six months later, your vector database doesn't know. It happily continues serving obsolete claims as ground truth.

We built PaperTrail to fundamentally rethink how software interacts with scientific literature.

Instead of another paper summarizer, PaperTrail is an autonomous, continuously evolving scientific intelligence engine.

In this article, I want to walk through the technical architecture, design philosophy, and lessons learned building PaperTrail with Next.js 14, TypeScript, local Ollama models, and a local-first SQLite/Supabase hybrid storage engine.

PaperTrail is built around a single thesis:

"Discover what research has established, what it challenges, what remains unexplored, and how the scientific evidence changes over time."

To deliver on this, we established ten non-negotiable architectural capabilities:

One of our strictest engineering rules is: No hallucinated consensus.

Every claim extracted by PaperTrail must adhere to a strict Zod schema. If the model cannot provide the exact supporting passage from the abstract or text, the extraction fails validation and falls back to a deterministic semantic parser.

Here is a snippet of our claim validation contract:

export const ClaimExtractionSchema = z. object ({ claims: z. array ( z. object ({ claim_text: z. string (). min ( 10 ), claim_type: z. enum ([ ' FINDING ', ' HYPOTHESIS ', ' METHODOLOGY ', ' LIMITATION ', ' BENCHMARK_RESULT ', ' THEORETICAL_BOUND ', ]), section: z. enum ([ ' Abstract ', ' Introduction ', ' Evaluation ', ' Conclusion ' ]), supporting_passage: z. string (). min ( 15 ), page_number: z. number (). int (). positive (). optional (), extraction_confidence: z. number (). min ( 0 ). max ( 1 ), }) ), experimental_context: z. object ({ model: z. string (). default ( ' Unspecified ' ), hardware: z. string (). default ( ' Unspecified ' ), dataset: z. string (). default ( ' Unspecified ' ), quantization: z. string (). default ( ' FP16 ' ), precision: z. string (). default ( ' Unspecified ' ), metrics: z. record ( z. string ()). default ({}), variables: z. record ( z. string ()). default ({}), }), limitations: z. array ( z. string ()). default ([]), }); Enter fullscreen mode Exit fullscreen mode By extracting both the claim and its experimental condition matrix (hardware, precision, model scale, metrics), we create the foundation for intelligent, context-aware comparison.

Traditional NLP pipelines treat contradiction detection as an NLI (Natural Language Inference) task: Premise vs. Hypothesis $\rightarrow$ Entailment, Neutral, or Contradiction.

In real science, this is catastrophically naive. Consider two papers:

A naive AI system flags these as a direct contradiction: Speedup vs. Throughput Drop.

PaperTrail executes a Context Comparison Matrix:

export interface ContextComparison { similarities: string []; differences: { variable: string; source_val: string; target_val: string; impact_level: ' LOW ' | ' MEDIUM ' | ' HIGH '; }[]; possible_explanations: string []; alternative_explanations: string []; remaining_uncertainty: string; is_genuine_contradiction: boolean; } Enter fullscreen mode Exit fullscreen mode When evaluated, PaperTrail identifies:

Most researchers know the dread of reading 30 papers, designing a novel experiment, and discovering two weeks later that a lab in Zurich published that exact experiment six months ago.

In PaperTrail, candidate research gaps are never taken at face value. They are subjected to a rigorous 6-Stage Adversarial Red-Team Audit:

[Candidate Gap Discovery] │ ▼ ┌──────────────────────────────────────────────┐ │ Stage 1: Hypothesis Decomposition │ │ Extract testable variables & parameters │ ├──────────────────────────────────────────────┤ │ Stage 2: Targeted Counter-Evidence Search │ │ Generate adversarial retrieval queries │ ├──────────────────────────────────────────────┤ │ Stage 3: Multi-Index Retrieval │ │ Query OpenAlex, Crossref, arXiv, TinyFish │ ├──────────────────────────────────────────────┤ │ Stage 4: Semantic Relevance & Overlap Audit │ │ Compare baseline methodology & datasets │ ├──────────────────────────────────────────────┤ │ Stage 5: Adversarial Refutation Scoring │ │ Compute falsification confidence (0.0-1.0) │ ├──────────────────────────────────────────────┤ │ Stage 6: Status Assignment │ │ SURVIVED (Validated) | FALSIFIED | PARTIAL │ └──────────────────────────────────────────────┘ Enter fullscreen mode Exit fullscreen mode Only when a candidate gap survives this adversarial gauntlet is it promoted to an actionable research gap. From there, PaperTrail automatically drafts an empirical experiment protocol complete with hardware requirements, dataset references, compute hours, and verification checklists.

We believe your research belongs to you—not locked inside our cloud database.

PaperTrail implements a dual-repository architecture:

┌───────────────────────────────┐ │ ResearchRepository │ │ ( Interface ) │ └───────────────┬───────────────┘ │ ┌────────────────┴────────────────┐ ▼ ▼ ┌───────────────────────────┐ ┌───────────────────────────┐ │ SQLiteResearchRepository │ │ SupabaseResearchRepository │ │ ( Node native DatabaseSync )│ │ ( PostgreSQL Cloud ) │ └───────────────────────────┘ └───────────────────────────┘ Enter fullscreen mode Exit fullscreen mode Using Node.js 22+'s native DatabaseSync ( node:sqlite ), PaperTrail operates entirely offline if desired, writing directly to.papertrail-data/papertrail.db with WAL mode enabled.

Every research workspace can be exported with one click into a structured, vendor-neutral research bundle:

papertrail-research/ ├── manifest.json # Cryptographic checksums, schema v1.0, timestamp ├── README.md # Complete human-readable investigation brief ├── papers/ # Verified metadata, OpenAlex/arXiv DOIs, abstracts ├── claims/ # Grounded assertions with verbatim supporting quotes ├── evidence/ # Disagreement matrices and apparent vs genuine verdicts ├── graph/ # Interactive topology nodes and edges (JSON) ├── research-gaps/ # 6-stage falsification audits and counter-evidence ├── experiments/ # Resource-evaluated execution protocols ├── datasets/ # Hugging Face & Zenodo references with license terms ├── timeline/ # Evidence evolution and tracking history └── provenance/ # Query origins and retrieval timestamps Enter fullscreen mode Exit fullscreen mode You can export this as a ZIP archive, BibTeX, JSON, or formatted Markdown, and import it into any other PaperTrail instance with zero data loss.

We made a conscious product design decision: PaperTrail should not look like another dark-mode SaaS dashboard with glowing neon gradients.

Science is deliberate, rigorous, and calm. We crafted a dedicated design language inspired by physical academic journals and high-end typography:

:root { --paper-bg: #F7F6F2; --paper-surface: #FDFCF9; --paper-surface-alt: #FAF8F3; --paper-ink: #252622; --paper-muted: #777873; --paper-border: #E3E1D9; --paper-accent: #B96645; --paper-data: #858B70; } Enter fullscreen mode Exit fullscreen mode The interface feels like reading a physical copy of Nature or Science paired with an interactive research workbench.

During our testing with local and remote Ollama models (specifically gpt-oss:120b-cloud ), we hit an interesting bottleneck.

When analyzing 8 papers simultaneously, each paper required claim extraction, experimental context parsing, pairwise contradiction checks, and gap discovery. Sending 15+ concurrent requests to a 120-billion parameter model over a cloud proxy resulted in request queueing that took 4+ minutes.

We solved this with a three-tiered pipeline architecture:

This reduced the end-to-end pipeline execution time from 4.5 minutes down to 14 seconds, delivering a fast, responsive UX without sacrificing scientific rigor.

PaperTrail is completely open source under the MIT license. Some of the features we're actively working on next:

If you're a researcher, engineer, or curious mind exploring literature:

Let me know in the comments: How do you currently track scientific contradictions and emerging research gaps in your field?