Abstract
Network intrusion detection models often achieve strong results within familiar benchmarks, but their reliability under simultaneous network-domain shift and previously unseen attack categories remains uncertain. We evaluate this problem using four NetFlow-v2 benchmark mirrors: NF-ToN-IoT, NF- BoT-IoT, and NF-UNSW-NB15 as development sources, and NF- CSE-CIC-IDS2018 as a frozen external evaluation network. A common 36-feature representation supports balanced empirical risk minimization (ERM), feature-augmented ERM, and aug- mented group distributionally robust optimization (GroupDRO); a histogram-gradient-boosted classifier provides a conventional baseline. Five random seeds, leave-one-domain-out development evaluation, source-only rejection calibration, and maximum softmax-probability and Mahalanobis detectors are reported. Mean leave-one-domain-out macro-F1 reaches only 0.2551 for GroupDRO, versus 0.2488 for ERM. On the stratified external sample, GroupDRO with Mahalanobis rejection reaches 0.2909 four class macro-F1 and 67.09% unknown recall, but rejects 35.09% of known traffic. Its raw external DoS and DDoS recalls are only 11.26% and 1.23%. Softmax-based rejection lowers known false rejection to 4.56%, but detects only 6.11% of unknown attacks. These results expose the practical tension between unknown-attack coverage, known-attack recognition, and calibrated alerting under cross-dataset shift. They constitute a reproducible negative-result analysis, not evidence of a deployment-ready intrusion detector.