research·linter

Lint report · stored run

Do Multimodal LLMs See Before They Read? Diagnosing Contextual Sycophancy

Yi-Cheng Lai, Hen-Hsen Huang open the source paper via the report id, not the extraction source: pdf IR v1.0 generated 2026-09-14 04:22:17 UTC
Lint another paper Raw report JSON
0errors
0warnings
0info
25claims read
517numbers extracted
24references
How well we read this paper 2 observations

These are notes about our reading of the paper, not about the paper. They affect how much weight the findings below can carry.

Note 2
  • numbers-are-identifier-fragments 12 extracted numbers are digits inside identifiers. e.g. '2.5' in 'GPT-5.1 Gemini 2.5 Qwen3-Thinking Claude', '2.5' in 'GPT-5.1 Gemini 2.5 Qwen3-Instruct Kimi-K', '2.5' in 'GPT-5.1 Gemini 2.5 Qwen3-Thinking Claude'. Rules that compare numbers across sections can report a contradiction between an identifier and a real value.
  • compile-warning references: the reference list carries no entry numbers; citation keys are enumeration order [1]..[N], not labels printed in the paper, so a key may not match the [N] used in the body
Some checks could not run
  • RL002 reported nothing, and it could not reach OpenAlex — so its silence here is not evidence that this paper is clean.
  • RL003 reported nothing, and it could not reach OpenAlex — so its silence here is not evidence that this paper is clean.

Everything else on this page was computed from the paper itself and is unaffected. This is a limit of this run, not a finding about the paper.

Some checks did not apply to this paper
  • RL005 did not apply to this paper — it links no repository we could read, so there was no released configuration to compare against. Its absence here says nothing about whether the paper and its code agree.
Compiler extraction log (1) 
references: the reference list carries no entry numbers; citation keys are enumeration order [1]..[N], not labels printed in the paper, so a key may not match the [N] used in the body

No findings on this paper.

That is a result, not a failure. The compiler read the paper without errors, and no rule found something it could prove with evidence — this product would rather say nothing than say something vague.

Read the extraction notes above before taking this as a clean bill of health: they say which checks were working with incomplete information.

Rule notes (33) 
RL001 funnel arithmetic: sentences_with_from=7 endpoint_pairs=2 claims_compared=0 emitted=0 [dropped: not-a-measurement=0]
RL001 funnel delta-column: structured_tables=11 with_exactly_one_delta_column=0 rows_checked=0 emitted=0
RL001 funnel cross-location: quant=517 prose=263 prose_with_metric=50 prose_with_dataset=3 prose_with_metric_and_dataset=0 distinct_buckets=0 emitted=0 [dropped: not-a-measurement=3 not-in-quote=8]
RL001 funnel identifier-drift: settings_checked=2 sections_stating_one_value=0 emitted=0
RL002 citation entailment: attempted 14 of 14 ranked cited claims (47 in-text markers from 24 bibliography entries); 8 cited works could not be identified, 3 had no retrievable text, 0 model quotes failed verbatim verification, 0 were topically unrelated to the retrieved text.
RL002 candidate ranking (why the top 14 were chosen): [11] [introduction, no DOI/arXiv id]; [22] [introduction, no DOI/arXiv id]; [10] [introduction]; [21] [introduction]; [15] [discussion, names an artefact (The GPT-4o), unusable title (identity fragile)]; [15] [experiment, absolutist wording, title/sentence overlap near zero, unusable title (identity fragile)]; [3] [results, title/sentence overlap near zero, unusable title (identity fragile)]; [15] [discussion, unusable title (identity fragile)]
RL002 label distribution over 3 classified citations: SUPPORTED=3
RL002 base-rate self-check: not applicable (3 classified, needs >=5).
RL002 attribution check (case B): 9 candidates name an artefact, 4 of them made the top-N and were checked against their cited source
RL002: 4 markers skipped because the claim is jointly attributed to several works
RL002: 3 enumerated-citation sentences skipped (they cite >=3 works, so the claim is about the set and cannot be refuted by any single cited work)
RL002: 5 duplicate candidates dropped (same reference and claim, or the same artefact attributed twice)
RL002: 8 cited works unresolvable (first 3: [{"key": "[15]", "title": "GPT-4o system card", "why": "bibliography title unusable for identity verification"}, {"key": "[15]", "title": "GPT-4o system card", "why": "bibliography title unusable for identity verification"}, {"key": "[3]", "title": "Qwen3-vl technical report", "why": "bibliography title unusable for identity verification"}])
RL003: 3 queries -> 179 unique works (arXiv search 104, category listings 0, scholarly graph 75 [crossref 75], 0 graph works matched to an arXiv id); source errors during retrieval: arXiv 0, graph 0
RL003: parsed the reference list of 11/12 comparable papers (1 unreadable) — that is the PeerUsage denominator
RL003: enriched 1 of 1 candidates with their own abstract (0 unavailable, 0 over budget); 0 dropped as published after this paper
RL003: scholarly graph for this run = crossref
RL003 funnel (636 candidates): already_used_by_the_paper -3, cited_only_from_related_work -128, is_a_dataset -5, is_a_metric -1, peer_usage_below_quorum -159, survey_or_artifact -45, title_names_no_method -294 || kept: citation_context_unknown +64 || survived-cheap-gates 1 -> with-abstract 1 -> comparable 1 -> above-threshold-0.65 0
RL003 candidate: 'Decoupled weight decay regularization' score 0.246 (T 0.00 D 0.50 M 0.33 P 0.27 R 0.00; 3/11 peers cite it) -> not reported
RL003: 1 comparable candidates all scored below 0.65 — nothing to report for this paper
RL004 funnel: components=11 contribution_backed=2 candidates=0 ablations_found=3 matched_by_ablation=0 unmatched=0 experiments=12
RL004 component M1 '998-case context-conditioned diagnostic benchmark': covered (filtered earlier)
RL004 component M2 'System-2 Visual Arbitration (S2VA)': covered (filtered earlier)
RL004 component M3 'Context-blind witness': covered (filtered earlier)
RL004 component M4 'System 2 Visual Arbiter': covered (filtered earlier)
RL004 component M5 'Leaky Witness': covered (filtered earlier)
RL004 component M6 'Witness-Only': covered (filtered earlier)
RL004 component M7 'Visual Supremacy Only': covered (filtered earlier)
RL004 component M8 'Evidence separation': covered (filtered earlier)
RL004 component M9 'CoVe-style verification proxy': covered (filtered earlier)
RL004 component M10 'Two-Call Describe–Answer': covered (filtered earlier)
RL004 component M11 'Single-Call Describe–Answer': covered (filtered earlier)
RL004 funnel: ranked=0 emitted=0 suppressed_by_cap=0 dropped_unverified_quote=0
Reading the report as data

This page is a rendering of the report the pipeline wrote. The JSON below is the stored artefact, unedited — the same document research-lint --format json produces. 

loading…