"I didn't Make the Micro Decisions": Measuring, Inducing, and Exposing Goal-Level AI Contributions in Collaboration

As large language models (LLMs) increasingly shape how users form, refine, and extend their goals, attributing contributions in human-AI collaboration becomes critical for users calibrating their own reliance and for evaluators assessing AI-assisted work. Yet existing methods focus on final artifacts, missing the process through which goals themselves are jointly shaped. We introduce a goal-level attribution framework, CoTrace, that decomposes explicit goals into verifiable requirements and traces both direct contributions and indirect influences across dialogue turns. Applying CoTrace to 638 real-world collaboration logs, we find that while models account for only 11-26% of goal-shaping contribution, they contribute substantially more on introducing lower-level concrete requirements, and make various kinds of indirect contributions. Through controlled simulations, we show that interaction design choices significantly affect model goal-shaping behavior. In a user study, exposing participants to goal-level analyses shifts their perceived contributions by nearly 2 points on a 5-point scale, revealing systematic miscalibration in how users understand their own AI-assisted work.

📋 한 줄 요약

**[Human-AI Collaboration / Attribution]** CoTrace가 explicit goal을 verifiable requirement로 분해해 dialogue turn 전반의 direct·indirect contribution을 추적, 638 협업 로그에서 LLM이 목표 형성의 11-26%만 기여하지만 lower-level requirement 도입에는 substantial 영향.

🎯 핵심 기여도

LLM이 사용자의 goal formation·refinement·extension에 점점 큰 영향을 미치는 가운데, human-AI collaboration에서 contribution 귀속이 사용자 reliance calibration과 evaluator의 AI-assisted work 평가에 critical해짐을 지적.
기존 방법이 final artifact에 집중해 goal이 jointly 형성되는 과정을 놓침을 진단.
CoTrace 도입 — goal-level attribution framework로 explicit goal을 verifiable requirement로 decompose, dialogue turn에 걸친 direct contribution·indirect influence를 추적.
638 real-world collaboration log에 CoTrace 적용, controlled simulation·user study로 효과·user 인식 변화 검증.

💡 핵심 아이디어

Human-AI collaboration의 contribution은 final artifact가 아니라 goal-shaping 과정의 verifiable requirement 단위로 분해·추적해야 정확히 귀속되며, 이를 통해 사용자가 자기 기여를 systematic 하게 miscalibration하고 있음을 노출할 수 있다.

🔬 기술적 접근법

**방법론**: CoTrace — goal-level attribution framework.
**핵심 기법**: (1) Explicit goal을 verifiable requirement로 decomposition, (2) Dialogue turn 전반의 direct contribution 추적, (3) Indirect influence(prompting·suggestion 등) 모델링, (4) 638 real-world collaboration log 분석, (5) Controlled simulation으로 interaction design 효과·user study로 인식 변화 측정.

📊 주요 결과

모델이 goal-shaping contribution의 11-26%만 차지.
Lower-level concrete requirement 도입에는 substantial 기여, 다양 indirect 기여.
Controlled simulation: interaction design 선택이 모델 goal-shaping 행동에 significant 영향.
User study: goal-level 분석에 노출되면 사용자의 perceived contribution이 5점 척도에서 약 2점 shift — systematic miscalibration 노출.

💭 의의 및 한계

**의의**: AI contribution 평가에 goal-level이라는 새 축 정립, dialogue 전반 indirect influence를 정량 포착, user의 self-attribution miscalibration을 정량 노출해 reliance·credit 평가 시스템 설계에 함의. **한계**: 638 log의 도메인 범위, verifiable requirement 분해의 평가자 의존, indirect contribution 정의의 task-specificity.

🚀 실용적 활용

AI-assisted writing·coding의 contribution attribution.
사용자 reliance calibration 도구.
학술·교육 evaluator의 AI-assisted work 평가.