Your Agents Are Aging Too: Agent Lifespan Engineering for Deployed Systems

Long-lived AI agents are increasingly deployed as persistent operational systems, yet they are still evaluated like freshly initialized models. Day-one benchmarks miss a basic systems question: how long does an agent remain reliable after deployment? Even when model weights are frozen, an agent's effective state keeps changing as it compresses interaction history, retrieves from a growing memory store, revises facts after updates, and undergoes routine maintenance. Reliability therefore becomes a lifespan property of the full agent harness, not only a snapshot property of the base model. We introduce AgingBench, a longitudinal reliability benchmark for agent lifespan engineering: measuring not only whether deployed agents degrade, but what form the degradation takes and where repair should target. AgingBench organizes agent aging into four mechanisms: compression aging, interference aging, revision aging, and maintenance aging. To diagnose these failures, AgingBench uses temporal dependency graphs and paired counterfactual probes that produce diagnostic profiles for the write, retrieval, and utilization stages of the memory pipeline. Across 7 scenarios, 14 models, multiple memory policies, and both runner-controlled and autonomous agents, over ~400 runs spanning 8 - 200 sessions show that agent aging is not one-dimensional: behavioral tests can remain clean while factual precision decays; derived-state tracking can collapse sharply within a single model; and the same wrong answer can require different repairs depending on what the diagnostic profile points to. These results suggest that reliable agent deployment requires lifespan evaluation, mechanism-level diagnosis, and stage-targeted repair, not only stronger day-one models.

📋 한 줄 요약

**[Agent Lifespan / Reliability Benchmark]** AgingBench가 deployed agent의 4 mechanism(compression·interference·revision·maintenance aging)을 temporal dependency graph·counterfactual probe로 진단, 7 시나리오·14 모델·~400 run에서 aging이 multi-dimensional·stage-specific repair 필요함을 입증.

🎯 핵심 기여도

Long-lived AI agent가 persistent operational system으로 점차 배포되지만 여전히 freshly initialized model처럼 평가받음을 지적 — Day-one benchmark가 배포 후 reliability 지속 시간이라는 기본 systems question을 놓침.
Model weight가 frozen이어도 agent의 effective state는 변화 — interaction history compression·growing memory store retrieval·fact revision·routine maintenance — 로 reliability가 base model snapshot 속성이 아닌 full agent harness의 lifespan 속성임을 정립.
AgingBench 도입 — agent lifespan engineering의 longitudinal reliability benchmark; deployed agent의 degradation 여부 뿐 아니라 형태·repair target을 측정.
4 mechanism으로 aging 조직 — compression aging, interference aging, revision aging, maintenance aging; 진단은 temporal dependency graph와 paired counterfactual probe로 memory pipeline의 write·retrieval·utilization stage에 대한 diagnostic profile 생성.

💡 핵심 아이디어

배포된 agent의 reliability는 day-one snapshot이 아닌 lifespan property이며, 4가지 aging mechanism으로 분해해 temporal dependency graph·counterfactual probe로 진단하면 behavioral test가 clean해도 factual precision이 decay하는 등 multi-dimensional 양상을 노출할 수 있어 stage-targeted repair가 가능하다.

🔬 기술적 접근법

**방법론**: AgingBench — longitudinal lifespan benchmark + mechanism diagnostics.
**핵심 기법**: (1) 4 aging mechanism 분류 — compression·interference·revision·maintenance, (2) Temporal dependency graph로 정보 흐름 추적, (3) Paired counterfactual probe로 write·retrieval·utilization stage별 diagnostic profile, (4) 7 시나리오·14 모델·다양 memory policy·runner-controlled·autonomous agent 통합, (5) 8~200 session span의 ~400 run으로 longitudinal 측정.

📊 주요 결과

7 시나리오·14 모델·다양 memory policy·~400 run(8~200 session) 결과 agent aging은 one-dimensional 아님.
Behavioral test는 clean하지만 factual precision이 decay할 수 있음.
Derived-state tracking이 단일 모델 내에서도 sharp 붕괴 가능.
동일한 wrong answer라도 diagnostic profile에 따라 다른 repair 필요.

💭 의의 및 한계

**의의**: AI agent 평가의 lifespan 차원 정립, 4 mechanism의 systematic 분해로 진단 가능, stage-targeted repair의 원칙적 근거 제공. **한계**: 7 시나리오·14 모델로 generalization 추가 검증 필요, longitudinal evaluation의 계산 비용 큼, memory pipeline 외 다른 component(planning 등) 미커버.

🚀 실용적 활용

Production AI agent의 lifecycle 관리.
Agent memory system 설계·debugging.
Long-running LLM agent의 reliability monitoring.