Temporal Contrastive Transformer for Financial Crime Detection: Self-Supervised Sequence Embeddings via Predictive Contrastive Coding

We introduce the Temporal Contrastive Transformer (TCT), a representation learning framework designed to capture contextual temporal dynamics in sequences of financial transactions. The model is trained using a self-supervised contrastive objective to produce embeddings that encode behavioral patterns over time, with the goal of supporting downstream fraud detection tasks. We evaluate TCT in a realistic setting by using the learned embeddings as input features to a gradient boosting classifier. Experimental results show that embeddings alone achieve meaningful predictive performance (AUC 0.8644), indicating that the model captures non-trivial temporal structure. However, when combined with domain-engineered features, no measurable improvement is observed over the baseline (AUC 0.9205 vs. 0.9245), suggesting that the learned representations largely overlap with existing feature abstractions. These findings position TCT as a promising representation learning approach that captures relevant behavioral signal, while highlighting the challenges of achieving additive value over strong domain features. The results reflect an intermediate stage in the development of temporal representation learning for financial crime detection and motivate further research on model architecture, training objectives, and integration strategies. At this early stage, achieving performance comparable to a strong feature-engineered baseline is itself a meaningful outcome, indicating that learned representations approximate domain-specific features without manual engineering. While not yet production-ready, these results point to a promising direction for reducing reliance on feature engineering in financial crime detection.

한 줄 요약

**[Fraud Detection / Self-Supervised]** Temporal Contrastive Transformer(TCT)가 금융 거래 sequence의 self-supervised contrastive 임베딩 학습으로 단독 AUC 0.8644 달성, 도메인 feature 결합 시 추가 향상은 미미(0.9205 vs 0.9245)로 representation 중첩 노출.

핵심 기여도

Temporal Contrastive Transformer(TCT) 제시 — 금융 거래 sequence의 contextual temporal dynamics를 캡처하는 representation learning 프레임워크.
Self-supervised contrastive objective로 학습해 시간에 걸친 behavioral pattern을 encoding하는 embedding 생성, downstream fraud detection task 지원 목표.
Gradient boosting classifier에 학습된 embedding을 입력 feature로 사용하는 realistic setting에서 평가 — embedding 단독으로 AUC 0.8644의 의미 있는 예측 성능 달성, 모델이 non-trivial temporal 구조 포착함을 시사.
그러나 domain-engineered feature와 결합 시 baseline 대비 measurable 향상 없음(AUC 0.9205 vs 0.9245) — 학습 representation이 기존 feature abstraction과 largely 중첩함을 시사. TCT를 의미 있는 behavioral signal을 캡처하는 promising representation learning 접근으로 자리매김하면서, 강한 domain feature 위에 additive value 달성의 도전 강조.

핵심 아이디어

금융 거래 fraud detection에서 self-supervised 표현 학습은 단독으로 강력한 신호를 잡지만, 잘 만들어진 도메인 feature와 중첩될 가능성이 크므로 representation·feature 통합 전략이 분리된 학습보다 중요하다.

기술적 접근법

**방법론**: TCT — Temporal Contrastive Transformer + predictive contrastive coding.
**핵심 기법**: (1) 금융 거래 sequence를 transformer로 인코딩, (2) Self-supervised contrastive objective로 behavioral pattern 학습, (3) Predictive contrastive coding으로 temporal 구조 캡처, (4) 학습 embedding을 gradient boosting classifier 입력 feature로 사용, (5) Domain-engineered feature와 결합한 hybrid 모델 비교.

주요 결과

TCT embedding 단독: AUC 0.8644 (의미 있는 예측 성능).
Embedding + domain feature: AUC 0.9205.
Baseline domain feature 단독: AUC 0.9245.
Additive value 부재 — representation이 도메인 feature와 largely 중첩.

의의 및 한계

**의의**: 금융 fraud detection의 self-supervised 표현 학습 baseline 제공, 강한 도메인 feature 위 additive value 달성의 도전 명시, 추후 architecture·objective·통합 전략 연구 동기. **한계**: 도메인 feature 대비 additive value 부재(현 시점 production-ready 아님), 단일 데이터셋 평가, contrastive objective design space 추가 탐색 필요.

실용적 활용

금융 거래 fraud detection의 표현 학습 baseline.
Feature engineering 의존도 감소 방향의 사전연구.
시계열 self-supervised 표현 일반 접근.