ReasearchReady: Side Hustle AI Challenge Week 2

Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle Submission 💸

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle.

ResearchReady gives researchers a private second look at a manuscript before peer review. Upload a LaTeX project, inspect source-linked findings, compare scientific claims with candidate manuscript evidence, and export a review report.

It combines reproducible submission checks with optional open-model review: broken citations and missing files are checked without inference; claim assessment and reviewer questions use an Ollama model. Every AI assessment requires human verification.

The potential customer is a small research lab, independent researcher, or research consultant handling unpublished drafts. The side-hustle hypothesis is paid local installation, model configuration, and support around an Apache-2.0 application. The value to test is whether a team would pay to keep control of its manuscripts while reducing review setup and preparation work. Customer demand, time savings, and willingness to pay are not yet validated; I am not claiming paying users.

The app includes two synthetic manuscripts: one with seeded defects and one without deterministic submission defects. Choose Try the synthetic sample, run submission checks, and open a finding to inspect its exact source. Select qwen3.5:9b and enable research integrity and reviewer simulation to reproduce the AI workflow.

Actual PDF export from the flawed sample · Raw flawed audit · Raw valid audit

An actual ResearchReady report from a synthetic manuscript, with summary counts and source-linked findings

In the final live smoke run, the flawed manuscript claimed a 15% improvement while reporting 70% versus 72% accuracy. ResearchReady linked that mismatch to the results source and questioned a universal-generalization claim supported only by one synthetic experiment.

The comparison manuscript described a two-percentage-point gain without claiming statistical significance. Its final run produced no AI warnings: one comparison had candidate evidence, while lack of independent corroboration for raw accuracy figures remained a neutral observation.

These examples were used during iteration. They are smoke tests, not a held-out accuracy benchmark. Timing includes application and polling overhead on a remote Spark reached through SSH; hardware was not independently measured. The run summary records the exact model digest and settings.

ResearchReady source and setup instructions

The workspace uses React and TypeScript, FastAPI, SQLite, and Ollama. Manuscript source stays in the backend's local project directory. The browser never receives the inference server address. JSON and paginated PDF exports support a review handoff.

The AI workflow uses qwen3.5:9b, Q4_K_M, served through Ollama. The server reports an Apache-2.0 model license. Claim quotes must match supplied raw source text; the application computes their locations. Lexical TF-IDF retrieval finds candidate evidence, excluding the claim's own occupied lines while preserving nearby source passages. Assessments and reviewer questions may cite only supplied passage IDs.

Live testing found problems that mocked tests could not. Default model thinking exhausted structured-output attempts. The runtime rejected bounded reviewer schemas with a grammar-parser error. Excluding an entire source chunk also hid evidence next to the claim and produced misleading results.

The fixes were to disable optional thinking for these structured requests, send a portable generation schema while retaining full Pydantic response validation, and retrieve nearby source spans without allowing a claim to support itself. Assessments now distinguish missing evidence from defects and descriptive percentage-point changes from statistical significance.

Thirty backend regression tests pass, including archive safety, exact quote grounding, evidence-location validation, status persistence, and report exports. Tests and the frontend production build also pass in an isolated installation without the private environment file. GitHub Actions runs the verification checks after pushing.

A research team should control which model and machine process its unpublished work. Compatible open-weight models can be served through the team's own Ollama endpoint, without a proprietary inference API key. A local endpoint keeps processing on the same machine; a trusted remote endpoint, such as the Spark used for this evaluation, receives analysis content.

Open code makes the prompts, parsing, retrieval, and validation inspectable. Teams can adapt checks, replace a model, and run the workspace on infrastructure they control. Local or self-hosted inference still has hardware and electricity costs, but does not require a proprietary per-request inference bill.

The present scope is LaTeX manuscript review. It does not fetch cited papers, interpret figure pixels, establish statistical validity, or replace scientific judgment. The next useful validation is an independently labeled evaluation and real researcher feedback.