{"paper":{"id":"2609.00048.pdf","title":"GUI-CC: Benchmarking Contextual Consistency of GUI World Models as Agent Environments","authors":["Lin Fu","Zheyuan Yang","Tianhui Zhang","Jinbiao Wei","Guo Gan","Boxu Liu","Yilun Zhao","Yu Rong"],"arxiv_id":"","url":"","source_kind":"pdf","ir_version":"1.0"},"generated_at":"2026-09-14T04:03:28.393664+00:00","counts":{"error":0,"warning":0,"info":0},"findings":[],"top_findings":[],"ir_summary":{"claims":20,"contributions":4,"method_components":12,"experiments":5,"baselines":17,"datasets":1,"metrics":12,"tables":5,"figures":22,"citations":27,"equations":0,"quantitative_results":476,"has_repository":false},"ir_health":{"observations":[{"code":"numbers-are-identifier-fragments","severity":"warning","message":"21 of 476 extracted numbers (4%) are digits inside identifiers, not measurements.","detail":"e.g. '8' in '8 | MobileWorld-html-8B w/ history | 61.', '2' in 'GPT Image 2 OpenAI | Proprietary (API) |', '3.1' in 'Gemini 3.1 Flash Image Google DeepMind |'. Rules that compare numbers across sections can report a contradiction between an identifier and a real value."},{"code":"compile-warning","severity":"note","message":"references: the reference list carries no entry numbers; citation keys are enumeration order [1]..[N], not labels printed in the paper, so a key may not match the [N] used in the body","detail":""},{"code":"compile-warning","severity":"note","message":"semantic extraction: dropped 1 items whose quote was not found in the paper","detail":""}],"errors":0,"warnings":1,"trustworthy":true},"diagnostics":{"source":"/home/oceann/workspace/research-linter/eval/papers/2609.00048.pdf","rules_run":["RL001","RL002","RL003","RL004"],"rules_skipped":[{"rule":"RL005","reason":"not-applicable"}],"rule_errors":{},"rule_findings":{"RL001":0,"RL002":0,"RL003":0,"RL004":0},"rule_seconds":{"RL001":0.01,"RL002":0.77,"RL003":95.19,"RL004":0.04},"dropped_no_evidence":0,"demoted_low_confidence":0,"ir_health":{"observations":[{"code":"numbers-are-identifier-fragments","severity":"warning","message":"21 of 476 extracted numbers (4%) are digits inside identifiers, not measurements.","detail":"e.g. '8' in '8 | MobileWorld-html-8B w/ history | 61.', '2' in 'GPT Image 2 OpenAI | Proprietary (API) |', '3.1' in 'Gemini 3.1 Flash Image Google DeepMind |'. Rules that compare numbers across sections can report a contradiction between an identifier and a real value."},{"code":"compile-warning","severity":"note","message":"references: the reference list carries no entry numbers; citation keys are enumeration order [1]..[N], not labels printed in the paper, so a key may not match the [N] used in the body","detail":""},{"code":"compile-warning","severity":"note","message":"semantic extraction: dropped 1 items whose quote was not found in the paper","detail":""}],"errors":0,"warnings":1,"trustworthy":true},"notes":["RL001 funnel arithmetic: sentences_with_from=2 endpoint_pairs=2 claims_compared=0 emitted=0 [dropped: not-a-measurement=0]","RL001 funnel delta-column: structured_tables=3 with_exactly_one_delta_column=0 rows_checked=0 emitted=0","RL001 funnel cross-location: quant=476 prose=69 prose_with_metric=0 prose_with_dataset=0 prose_with_metric_and_dataset=0 distinct_buckets=0 emitted=0 [dropped: not-a-measurement=0 not-in-quote=0]","RL001 funnel identifier-drift: settings_checked=2 sections_stating_one_value=0 emitted=0","RL002 citation entailment: attempted 9 of 9 ranked cited claims (49 in-text markers from 27 bibliography entries); 6 cited works could not be identified, 3 had no retrievable text, 0 model quotes failed verbatim verification, 0 were topically unrelated to the retrieved text.","RL002 candidate ranking (why the top 9 were chosen): [8] [introduction, title/sentence overlap near zero, no DOI/arXiv id]; [24] [introduction, title/sentence overlap near zero, no DOI/arXiv id]; [14] [introduction, no DOI/arXiv id]; [9] [introduction, no DOI/arXiv id]; [23] [introduction, no DOI/arXiv id]; [3] [experiment, names an artefact (Eval Consistency Eval\nWMA), title/sentence overlap near zero, no DOI/arXiv id]; [8] [experiment, names an artefact (CUWM), title/sentence overlap near zero, no DOI/arXiv id]; [24] [experiment, names an artefact (MobileWorld), title/sentence overlap near zero, no DOI/arXiv id]","RL002 attribution check (case B): 3 candidates name an artefact, 3 of them made the top-N and were checked against their cited source","RL002: 14 enumerated-citation sentences skipped (they cite >=3 works, so the claim is about the set and cannot be refuted by any single cited work)","RL002: 6 cited works unresolvable (first 3: [{\"key\": \"[8]\", \"title\": \"Computer-using world model\", \"why\": \"no work record matched the bibliography title\"}, {\"key\": \"[24]\", \"title\": \"How mobile world model guides gui agents?\", \"why\": \"no work record matched the bibliography title\"}, {\"key\": \"[9]\", \"title\": \"Generative visual code mobile world models\", \"why\": \"no work record matched the bibliography title\"}])","RL003: 3 queries -> 117 unique works (arXiv search 50, category listings 0, scholarly graph 67 [crossref 67], 0 graph works matched to an arXiv id); source errors during retrieval: arXiv 0, graph 0","RL003: parsed the reference list of 12/12 comparable papers (0 unreadable) — that is the PeerUsage denominator","RL003: enriched 8 of 15 candidates with their own abstract (0 unavailable, 7 over budget); 0 dropped as published after this paper","RL003: scholarly graph for this run = crossref","RL003 funnel (847 candidates): already_used_by_the_paper -3, cited_only_from_related_work -235, peer_usage_below_quorum -165, shares_no_dataset_metric_or_close_task -1, survey_or_artifact -192, title_names_no_method -237 || kept: citation_context_unknown +31 || survived-cheap-gates 15 -> with-abstract 8 -> comparable 14 -> above-threshold-0.65 0","RL003 candidate: 'UI-TARS' score 0.617 (T 0.50 D 1.00 M 0.33 P 0.50 R 1.00; 6/12 peers cite it) -> not reported","RL003 candidate: 'Mobile-agent-v3. 5' score 0.600 (T 0.50 D 1.00 M 0.33 P 0.42 R 1.00; 5/12 peers cite it) -> not reported","RL003 candidate: 'Mobile-agent-v3' score 0.583 (T 0.50 D 1.00 M 0.33 P 0.33 R 1.00; 4/12 peers cite it) -> not reported","RL003 candidate: 'GUI-R1' score 0.567 (T 0.50 D 1.00 M 0.33 P 0.25 R 1.00; 3/12 peers cite it) -> not reported","RL003 candidate: 'Os-genesis' score 0.567 (T 0.50 D 1.00 M 0.33 P 0.25 R 1.00; 3/12 peers cite it; title-only, no abstract available) -> not reported","RL003: 15 comparable candidates all scored below 0.65 — nothing to report for this paper","RL004 funnel: components=12 contribution_backed=4 candidates=0 ablations_found=1 matched_by_ablation=0 unmatched=0 experiments=5","RL004 component M1 'GUI-CC': covered (filtered earlier)","RL004 component M2 'Offline Reference-Action Track': covered (filtered earlier)","RL004 component M3 'Online Agent-Loop Track': covered (filtered earlier)","RL004 component M4 'semantic schema': excluded:not-a-claimed-contribution","RL004 component M5 'Transition Fidelity': covered (filtered earlier)","RL004 component M6 'Transition Plausibility': covered (filtered earlier)","RL004 component M7 'Contextual Consistency': excluded:statistical-or-evaluation-procedure","RL004 component M8 'Task Progress': covered (filtered earlier)","RL004 component M9 'Single-Step Plausibility Verification': excluded:not-a-claimed-contribution","RL004 component M10 'milestone sequence': excluded:not-a-claimed-contribution","RL004 component M11 'VLM judges': excluded:not-a-claimed-contribution","RL004 component M12 'history-conditioned input': covered (filtered earlier)","RL004 funnel: ranked=0 emitted=0 suppressed_by_cap=0 dropped_unverified_quote=0"],"missing_core_rules":[],"seconds":112.86,"llm":{"calls":0,"cache_hits":1,"errors":0,"prompt_chars":0},"sources":{"arxiv":{"requests":19,"errors":0,"cache_hits":26,"available":1},"openalex":{"requests":1,"errors":0,"cache_hits":0,"available":0,"unavailable_reason":"OpenAlex free daily budget exhausted (resets midnight UTC)"},"crossref":{"requests":1,"errors":0,"cache_hits":11,"available":1},"s2":{"requests":0,"errors":0,"cache_hits":0,"available":1},"short_circuited":["crossref","openalex","s2"]},"cache":{"hits":38,"misses":21}},"compile_warnings":["references: the reference list carries no entry numbers; citation keys are enumeration order [1]..[N], not labels printed in the paper, so a key may not match the [N] used in the body","semantic extraction: dropped 1 items whose quote was not found in the paper"]}