Cocoa: Co-Planning and Co-Execution with AI Agents
Paper URL: arXiv:2412.10999
arXiv subjects: cs.HC (Human-Computer Interaction), cs.AI (Artificial Intelligence)
AI agents can now search for papers, summarize literature, and carry out multi-step tasks. The difficult question is how people should remain involved while that work unfolds. Cocoa argues that chat is often too coarse a control surface: users can ask for a result and react to it, but have few natural ways to divide work with an agent or revise its direction halfway through.
Feng and colleagues introduce Cocoa, a document-based interface for scientific research tasks. Its central feature is an interactive plan embedded in the document. The agent proposes a plan, but the researcher can edit, add, delete, reorder, or reassign its steps. A step can belong to the agent or the researcher. The plan can run continuously or one step at a time, and the researcher can edit intermediate outputs before later steps use them as context. Planning and execution can therefore alternate instead of appearing as separate phases.
This is the paper’s main contribution: a shared representation of tasks, roles, and progress that makes agency visible and negotiable. The authors call the two kinds of interaction co-planning and co-execution. Co-planning lets the person and agent shape the work together; co-execution lets them complete different parts of it while revising the plan as evidence or understanding changes. The design draws on computational notebooks, where people run and inspect work incrementally rather than treating a program as a single opaque operation.
The researchers developed the system through a formative study with nine researchers. They then compared Cocoa with a chat interface using the same GPT-4o-based research agent in a within-subjects lab study of 16 PhD students and postdoctoral researchers. Participants used each system on open questions or unfinished tasks from their own research documents. A seven-day deployment with seven participants examined how Cocoa fit into ongoing research work.
The strongest result concerns steerability. Participants rated Cocoa as more steerable than chat: the median rating was 4 for Cocoa and 3 for the baseline (p = .005). Cocoa did not show a significant ease-of-use disadvantage, although this should not be read as proof that the systems are equivalent. The interaction data also suggest a meaningful workflow shift. Participants spent less time passively inspecting output in Cocoa (33.6% rather than 54.1% in chat) and more time editing, assigning, and redirecting work. They edited steps, reassigned responsibility, added steps, and triggered replanning rather than merely asking the agent to revise a completed answer.
The field study makes the allocation of agency more concrete. When working on their own projects, participants tended to keep strategic decisions, argument development, deep reading, experiments, and judgments about research novelty for themselves. They assigned lower-risk, more bounded work to the agent, such as searching for literature or expanding preliminary ideas. This is a useful account of delegation: a good human-agent workflow is not necessarily one in which the agent does the most work. It is one in which people can decide which work requires their expertise.
The paper is careful but its evidence has clear limits. It shows higher perceived and interactional steerability, not better research. It does not demonstrate that Cocoa improves the correctness, novelty, quality, or productivity of research outputs. The samples are small and mostly drawn from CS or adjacent fields, while the baseline differs from Cocoa in several interface affordances at once. A document is also a poor fit for branching exploration or parallel plan versions, a limitation the authors acknowledge. Finally, steering depends on users being able to recognize weak intermediate outputs; editable plans do not by themselves solve factual errors or misplaced trust in an agent.
For the design of AI agents, Cocoa is a strong argument for moving beyond the one-shot prompt-and-response model. Long-running work benefits from intermediate points where people can inspect, revise, and reclaim responsibility. The next study should test the outcome that matters most: whether these additional control points produce better research artifacts, rather than simply more interaction or a stronger feeling of control. It should also isolate the value of each feature—editable plans, step assignment, incremental execution, output editing, and replanning—and compare document-based workflows with interfaces that better support branching inquiry.