research·linter

A linter for scientific papers

Find what's wrong with your research before reviewers do.

  • paper.pdf
  • PaperIR
  • rules
  • warnings, with evidence

Research Linter is not an AI reviewer and not a proofreader. It is a compiler: your paper is compiled once into a structured representation, and every check is an independent rule over that structure. A warning ships only if it carries a quote you can check.

Lint your paper Read a real report
5 rules over the compiled IR 0 of 4 findings shown without evidence arXiv id or PDF in
Library stacks at night: a corridor of shelves lit by a single row of hanging lamps.
Every shelf, catalogued. A reviewer reads a paper once and remembers what struck them. A linter reads every number, every table and every reference the same way, every time. Photograph: Pexels, free to use.

A real finding

This is the whole product, in one card.

Read the quote before you read the claim. Every warning this tool emits looks like this, and every one of them came out of a real run on a real arXiv paper — nothing on this page was written by hand.

RL004 Missing ablation Warning

No ablation isolates the Semantic Anchor Generation via LLM Augmented Report Synthesis

at Method confidence 0.75 refs M2, K2

The paper introduces Semantic Anchor Generation via LLM Augmented Report Synthesis as one of its own contributions (it is referred to by a numbered contribution), but no reported experiment removes or replaces it. The ablation labels actually present are 'similarity-aware negative sampling', 'FAISS-based distant negatives -> random in-batch negatives', 'Masked Reconstruction Regularizer (LMSE)', 'LMSE', 'audio encoder adaptation', 'Frozen Audio Backbone', 'MedSigLIP text encoder', 'MedSigLIP -> BERT', 'similarity-aware negative sampling (k=10)', 'negative sampling k=10 -> K=100', 'pre-trained audio encoder', 'pre-trained audio encoder -> audio encoder from scratch', 'Audio Encoder from Scratch', 'Random Negatives', 'No LMSE', 'BERT Text Encoder', 'Negative Sampling K=100', and none of them corresponds to Semantic Anchor Generation via LLM Augmented Report Synthesis, so its individual contribution is not isolated by the experiments as reported.

Evidence — 2 items
Quoted from this paper Method
Semantic Anchor Generation via LLM Augmented Report Synthesis Contrastive alignment requires semantically rich, paired text for each audio sample.
Search that found nothing every experiment's ablated components and ablation label
Searched all 12 extracted experiment(s) for an ablation of 'Semantic Anchor Generation via LLM Augmented Report Synthesis'. Ablation labels found: 'similarity-aware negative sampling', 'FAISS-based distant negatives -> random in-batch negatives', 'Masked Reconstruction Regularizer (LMSE)', 'LMSE', 'audio encoder adaptation', 'Frozen Audio Backbone', 'MedSigLIP text encoder', 'MedSigLIP -> BERT', 'similarity-aware negative sampling (k=10)', 'negative sampling k=10 -> K=100', 'pre-trained audio encoder', 'pre-trained audio encoder -> audio encoder from scratch', 'Audio Encoder from Scratch', 'Random Negatives', 'No LMSE', 'BERT Text Encoder', 'Negative Sampling K=100'. No label matches 'Semantic Anchor Generation via LLM Augmented Report Synthesis' or its tokens (anchor, augment, generation, llm, report, semantic, synthesi).
ablations_found
17.0

What to do Add a run with Semantic Anchor Generation via LLM Augmented Report Synthesis removed or replaced by a simpler alternative, so the claim that it contributes can be separated from the rest of the method.

From the full report on “Zero-Shot Respiratory Sound Classification through LLM-Augmented Audio-Text Alignment”

not this
“Consider adding more baselines to strengthen your evaluation.” — advice you cannot check and cannot act on.
this
A named method, the same task, the same dataset, the same metric, and the overlap scores that establish it — so you can disagree with the finding on the evidence rather than on trust.
why it matters
Three findings you would actually fix beat thirty you would skim. The pipeline ranks everything and shows you the top three first.

Example reports

Reports we already ran, open for reading.

These are stored runs of the real pipeline over papers from our corpus. Nothing is a screenshot: every one opens the report the pipeline wrote, with its findings, its evidence and its diagnostics.

Zero-Shot Respiratory Sound Classification through LLM-Augmented Audio-Text Alignment

0 error · 2 warning · 0 info
open report

RePro: Proof-Verified Benchmark Rewriting for Reliable Evaluation of LLM Mathematical Problem Solving

0 error · 2 warning · 0 info
open report

trajectory-judge: What Outcome-Only LLM Judges Miss on Agent Trajectories

0 error · 0 warning · 0 info
open report

How it works

Compile it, then lint it.

Asking a model “what is wrong with this paper?” produces plausible prose you cannot verify. Compiling the paper first produces a structure that code can check and that you can inspect.

STEP 01

Compile to PaperIR

Text, sections, tables, references, numbers and the sentence each number came from are extracted into one typed structure. Numbers, citation identity and table geometry are pulled out without a model, so they can be checked exactly. One model pass reads the whole paper for meaning — claims, contributions, baselines — and every quote it returns must appear verbatim in the source or it is dropped. Rules that need a judgement of meaning make their own calls on top of that, so a run costs one call plus however many the paper's citations require; the report prints the count it actually made.

STEP 02

Run the rules

Each rule is an independent function over that structure: does this number agree with that table, does this citation support this claim, is this claimed component ever ablated. Rules never see the PDF and never call a model unless the question is genuinely one of meaning. Arithmetic happens in Python, always.

STEP 03

Keep only what is checkable

Before anything reaches you, every finding passes an admissibility gate: it must carry at least one piece of evidence, and a claimed error must clear 0.7 confidence or it is demoted to a warning. Both are enforced in the pipeline on every run — the rule catalogue lists exactly what the gate does and, just as importantly, what it does not. Findings that fail are counted and reported, so a thin run looks thin instead of looking clean.

Rule catalogue

5 checks, each with one job.

RL001

Result consistency

no model offline

Do the paper's numbers agree with each other?

  • The same quantity reported with different values across the abstract, body, tables and appendix.
  • Arithmetic that does not follow: the text claims +8.3% but the paper's own table shows 74.2 to 80.1.
  • An identifier — dataset, metric, seed, sample size — named one way in the text and another in a table or caption.
  • Cross-references that dangle: a table referenced before it exists, a figure never referenced at all.
RL002

Citation entailment

uses a model network

Does the cited work actually say what the claim says?

  • Scope overgeneralisation: a claim that a cited paper showed a general result when it showed one on two synthetic benchmarks.
  • Claims attached to a citation the cited abstract does not support.
  • Claims the cited work directly contradicts.
RL003

Missing baseline

uses a model network

Is there a comparable method the paper should have run against?

  • A published method that evaluates the same task on the same dataset with the same metric and is missing from the comparison.
  • Above all when the paper's own related-work cluster benchmarks against it.
RL004

Missing ablation

no model offline

Is every claimed contribution actually isolated?

  • A component presented as a contribution that no reported ablation independently removes or replaces.
RL005

Paper / code mismatch

no model network

Does the released code do what the paper says it does?

  • A hyperparameter the paper states explicitly — batch size, learning rate, optimiser, epochs, seed — that the configuration in the paper's own linked repository disagrees with.
  • A configuration the repository leaves ambiguous across several files, reported as a question rather than as a mismatch.

Read the full rule contracts

Why you can check it

Models extract and reason. Code verifies.

This is the one engineering rule the whole product is built around. Where it is followed, a finding is arithmetic and cannot be argued with. Where it cannot be followed, the uncertainty is printed on the finding.

arithmetic
Recomputed in Python from the paper's own table cells. Never asked of a model, never estimated.
citation identity
DOI, arXiv id and year parsed with regular expressions, so a reference is resolved or it is not — there is no third state.
quotes
A model-produced quote must be an exact substring of the text the model was shown. If it is not, the finding is discarded rather than softened.
absence
“This paper has no ablation” is only reported together with the search that established it, and never when the extraction itself reported a failure.
failure
A run that could not reach the network, skipped a rule or dropped a finding says so on the report. A quiet empty report and a broken run must not look the same.
A drawer of a library card catalogue pulled open, showing handwritten catalogue cards with a bibliographic reference.
Catalogued, card by card Every claim in a paper points at a record. RL002 checks that the record actually says what the claim says. Wikimedia Commons, CC BY-SA 4.0
A stack of punched cards bound with a red elastic band, their printed numeric rows visible.
A structure a machine can read PaperIR is the same idea: fix the shape of the data first, then every check becomes mechanical. Wikimedia Commons, CC BY-SA 3.0
A compositor's type case seen from above, filled with rows of metal letter sorts.
Set once, checked character by character Typesetting was verified piece by piece before printing. So is a paper, before a reviewer does it for you. Wikimedia Commons, CC BY-SA 4.0
Rows of wooden catalogue drawers in a library, each with a brass label holder and a round handle.
Findable, or it does not exist A finding you cannot trace back to a source is an opinion. Wikimedia Commons, CC BY-SA 4.0

Put a paper through it.

Upload a PDF or give an arXiv id. Compiling a paper takes a couple of minutes — you will see each stage as it happens, and the report is a URL you can send to a co-author.

Lint a paper See what it checks