Abstract
Automated kidney CT classification often reports very high accuracy, but slice-based datasets can contain near-duplicate images from the same scan. When those slices are separated by random image-level splits, test results can be severely inflated. This study quantifies that effect and proposes a reliable evaluation protocol for four-class kidney CT classification. Using the CT-KIDNEY dataset which has 12,446 images, we perform two-stage near-duplicate auditing with perceptual hashing for candidate screening and ResNet-18 embeddings with cosine similarity for verification. Images are grouped into similarity clusters and assigned to train, validation, and test sets with cluster-disjoint splitting so that no cluster appears in more than one partition. ResNet-34 is evaluated under both random and cluster-disjoint splits to measure leakage effects, while ConvNeXt-Base is trained and tested only under the strict protocol. Grad-CAM is used to inspect model attention on held-out test images. The audit shows that 99.98% of images belong to near-duplicate clusters. Under random splitting, ResNet-34 reaches 100.00% test accuracy, but under cluster-disjoint evaluation the same model drops to 83.22%. Under the same reliable protocol, ConvNeXt-Base achieves 96.98% accuracy with macro F1-score 93.89%. These results show that random slice-level splitting can substantially overestimate kidney CT performance and that cluster-disjoint evaluation provides a more trustworthy benchmark for clinically meaningful generalization.