TabPFN and TabICL vs. tuned XGBoost: the model that doesn't train won 14/14
Researchers compared the performance of TabPFN and TabICL, two neural network approaches that require no training, against a finely tuned XGBoost model across fourteen benchmark datasets. The untuned models achieved perfect results, winning every comparison. The findings suggest that these zero‑training methods can match or exceed traditional gradient‑boosted trees in accuracy on the tested tasks.