How data is split can swing encrypted traffic classifier scores by 40 points
Standard random data splits inflate accuracy to near-perfect levels, while session-disjoint splits reveal performance at chance.
Independent
Mayan Sharma · Kriti Saini · Devansh Behl
Research Digest··2 min read
The authors show that the evaluation protocol, specifically how packet flows are partitioned into training and test sets, can change reported macro-F1 from 0.992 to 0.567 on the same model and code.
Using two public corpora of encrypted traffic (including a month of backbone QUIC traffic), the authors re-evaluated a common flow-level classifier under five split protocols: random, temporal, client-disjoint, session-disjoint, and server-disjoint.
