iPrefBench: A User-Centric Benchmark for Personalized Mobile Use Agents
iPrefBench is a benchmark for evaluating whether personal mobile-use agents can infer missing requirements from a user's activity history and ask for information only the user can provide. It runs in an executable iOS environment with an interactive simulated user, grounded in a persona with 3,051 past activity events. It contains 130 underspecified tasks across 26 apps. Evaluation of 7 frontier agents shows high action completion but low full task success, with particular difficulty on ask-only information.
One primary source; other links are context, not independent corroboration.
- Primary sourceHugging Face Daily Papers
- Discovery linkHugging Face
- Linked in sourceView arXiv page
- Linked in sourceView PDF
The paper page is linked. Peer review and findings have not been independently checked by AINewsIQ.
Open full article view →- Original paper+45
- Fetched source page+30
- No editor link review+0
- Original non-discussion link+5
This measures the stored source trail and editorial steps. It does not certify accuracy, peer review, or independent corroboration. Popularity adds no points.
View full scoring →Read the scoring rubric →What the source says
iPrefBench is a benchmark for evaluating whether personal mobile-use agents can infer missing requirements from a user's activity history and ask for information only the user can provide. It runs in an executable iOS environment with an interactive simulated user, grounded in a persona with 3,051 past activity events. It contains 130 underspecified tasks across 26 apps. Evaluation of 7 frontier agents shows high action completion but low full task success, with particular difficulty on ask-only information.
Key findings from the stored brief
- Agents reached up to 91.6% action completion but only up to 47.7% full task success.
- Personalization lags behind execution in the evaluated agents.
- Agents especially struggle to request enough necessary information on ask-only requirements.
- Each omitted requirement is by construction either history-inferable or ask-only.
Source claims not independently checked; results cover 7 agents in a simulated iOS environment with a simulated user.
This is a source-based abstract, not independent validation of the source's claims.
Evidence & sources
These links show where the signal came from and what the source page linked to. A linked page is context, not independent corroboration.
Related archive coverage
These relationships record a citation or topic overlap, not confirmation of the finding.
The paper page is linked. Peer review and findings have not been independently checked by AINewsIQ.
Timeline & updates
- Signal dated by source or discovery feed
This timestamp places the signal in the radar. It may be the discovery post date rather than the primary page's first publication date.
- Shared brief stored
AINewsIQ prepared one abstract from fetched source page.
A publication revision history is not stored for this signal. New engagement snapshots do not mean the source itself changed.
Why this may matter
Computer-use agents may execute tasks well yet fail to act for a specific user. Benchmarks that measure both inference from history and appropriate questioning can help developers build assistants that know their users and check in when needed.
Limits and open questions
Source claims not independently checked; results cover 7 agents in a simulated iOS environment with a simulated user.
Organization-specific financial, security, and operational effects require the organization's own context and review.
Discussion
Ask about the source, add context, or point readers to a relevant follow-up.
Discussion
Comments are reviewed before publication. Discuss the evidence, not the person.
No published comments yet.
Reader discussion opens with accounts. You can read the original source and published comments now.