{"paper":{"id":"2609.00063.pdf","title":"Medical Causal Hypothesis Verification with Large Language Models","authors":["Safiyyah Ahmed","Abrar Ansari","Md Aminul Islam","Elena Zheleva"],"arxiv_id":"","url":"","source_kind":"pdf","ir_version":"1.0"},"generated_at":"2026-09-14T04:14:58.736075+00:00","counts":{"error":0,"warning":0,"info":0},"findings":[],"top_findings":[],"ir_summary":{"claims":24,"contributions":3,"method_components":7,"experiments":6,"baselines":0,"datasets":0,"metrics":11,"tables":2,"figures":2,"citations":8,"equations":0,"quantitative_results":192,"has_repository":false},"ir_health":{"observations":[{"code":"compile-warning","severity":"note","message":"semantic extraction: dropped 4 items whose quote was not found in the paper","detail":""}],"errors":0,"warnings":0,"trustworthy":true},"diagnostics":{"source":"/home/oceann/workspace/research-linter/eval/papers/2609.00063.pdf","rules_run":["RL001","RL002","RL003","RL004","RL005"],"rules_skipped":[],"rule_errors":{},"rule_findings":{"RL001":0,"RL002":0,"RL003":0,"RL004":0,"RL005":0},"rule_seconds":{"RL001":0.01,"RL002":94.35,"RL003":0.47,"RL004":0.01,"RL005":0.49},"dropped_no_evidence":0,"demoted_low_confidence":0,"ir_health":{"observations":[{"code":"compile-warning","severity":"note","message":"semantic extraction: dropped 4 items whose quote was not found in the paper","detail":""}],"errors":0,"warnings":0,"trustworthy":true},"notes":["RL001 funnel arithmetic: sentences_with_from=5 endpoint_pairs=1 claims_compared=0 emitted=0 [dropped: not-a-measurement=0]","RL001 funnel delta-column: structured_tables=2 with_exactly_one_delta_column=0 rows_checked=0 emitted=0","RL001 funnel cross-location: quant=192 prose=68 prose_with_metric=55 prose_with_dataset=0 prose_with_metric_and_dataset=0 distinct_buckets=0 emitted=0 [dropped: not-a-measurement=0 not-in-quote=41]","RL001 funnel identifier-drift: settings_checked=2 sections_stating_one_value=0 emitted=0","RL002 citation entailment: attempted 5 of 5 ranked cited claims (15 in-text markers from 8 bibliography entries); 3 cited works could not be identified, 2 had no retrievable text, 0 model quotes failed verbatim verification, 0 were topically unrelated to the retrieved text.","RL002 candidate ranking (why the top 5 were chosen): [8] [introduction, absolutist wording, title/sentence overlap near zero, no DOI/arXiv id]; [3] [introduction, absolutist wording, no DOI/arXiv id]; [2] [introduction, absolutist wording, no DOI/arXiv id]; [4] [introduction, no DOI/arXiv id]; [6] [experiment, names an artefact (Label Studio), no DOI/arXiv id]","RL002 attribution check (case B): 1 candidates name an artefact, 1 of them made the top-N and were checked against their cited source","RL002: 1 citations skipped because the sentence reports this paper's own results (the citation there is a comparison pointer, not evidence for the sentence)","RL002: 1 markers skipped because the claim is jointly attributed to several works","RL002: 3 cited works unresolvable (first 3: [{\"key\": \"[8]\", \"title\": \"Causal parrots: Large language models may talk causality but are not causal\", \"why\": \"no work record matched the bibliography title\"}, {\"key\": \"[3]\", \"title\": \"Causal reasoning and large language models: Opening a new frontier for causality\", \"why\": \"no work record matched the bibliography title\"}, {\"key\": \"[6]\", \"title\": \"Label Studio: Data labeling software\", \"why\": \"no)","RL003: 2 queries -> 149 unique works (arXiv search 99, category listings 0, scholarly graph 50 [crossref 50], 0 graph works matched to an arXiv id); source errors during retrieval: arXiv 0, graph 0","RL003: parsed the reference list of 0/0 comparable papers (0 unreadable) — that is the PeerUsage denominator","RL003: could read the reference list of only 0 comparable papers (need 4) — peer usage unavailable, no findings","RL004 funnel: components=7 contribution_backed=2 candidates=0 ablations_found=0 matched_by_ablation=0 unmatched=0 experiments=6","RL004 component M1 'Evaluation framework for causal hypothesis verification': excluded:is-the-whole-method","RL004 component M2 'Standardized prompt template': excluded:evaluation-infrastructure-or-packaging","RL004 component M3 'Automated pipeline for collecting and parsing LLM responses': excluded:not-a-claimed-contribution","RL004 component M4 'Human annotation of hypothesis and supporting evidence': excluded:not-a-claimed-contribution","RL004 component M5 'Evaluation metrics for hypothesis classification and evidence grounding': excluded:not-a-claimed-contribution","RL004 component M6 'Label Studio': excluded:not-a-claimed-contribution","RL004 component M7 '17 causal hypotheses validated by four expert clinicians': excluded:not-a-claimed-contribution","RL004 funnel: ranked=0 emitted=0 suppressed_by_cap=0 dropped_unverified_quote=0","RL005: repo https://github.com/HumanSignal/label-studio unusable (stopped early after 400 files / 2071188 bytes)"],"missing_core_rules":[],"seconds":99.83,"llm":{"calls":0,"cache_hits":1,"errors":0,"prompt_chars":0},"sources":{"arxiv":{"requests":5,"errors":0,"cache_hits":15,"available":1},"openalex":{"requests":1,"errors":0,"cache_hits":0,"available":0,"unavailable_reason":"OpenAlex free daily budget exhausted (resets midnight UTC)"},"crossref":{"requests":5,"errors":0,"cache_hits":2,"available":1},"s2":{"requests":0,"errors":0,"cache_hits":0,"available":1},"short_circuited":["crossref","openalex","s2"]},"cache":{"hits":18,"misses":11}},"compile_warnings":["semantic extraction: dropped 4 items whose quote was not found in the paper"]}