Predictive maintenance using administrative work orders: A machine learning framework for failure criticality classification.

Abstract

This study evaluates the feasibility of predictive maintenance using only historical correc-tive maintenance records from a food industry plant, without sensor data. The dataset comprised 6,731 failure events across 664 machines, with nine operational variables. Failure criticality was classified into low (≤23 min), medium (24–44 min), and high (>44 min) based on downtime percentiles. Three tree based models (Random Forest, XGBoost, LightGBM) and a multilayer perceptron were developed. Hyperparameter optimization was performed using RandomizedSearchCV and Optuna, the latter with a custom gap penalized objective function to mitigate overfitting. A chronological split (70/15/15) respected temporal order. The best model, XGBoost with gap penalized Optuna, achieved a test Macro F1 of 0.3829 and a weighted F1 of 0.4871, with a minimal generalization gap. The neural network obtained the highest weighted F1 (0.5512) but failed on the medium criticality class. Results confirm that administrative records alone have a structural pre-dictive ceiling, and that explicit gap penalization is effective for boosting models. The proposed framework enables systematic comparison of tree based and neural models for failure criticality classification using only work order logs. While statistical methods such as ANOVA can establish whether significant differences exist between operational groups, they do not provide a predictive framework capable of classifying new failure events at the moment they occur. Conversely, machine learning models are designed to identify and generalize patterns from historical data, but do not inherently explain the statistical structure of the underlying variables. This study addresses both dimensions in a complementary fashion: ANOVA and post-hoc Tukey HSD tests are used to statistically validate the relevance of machine identity, work shift interaction, and technician count on equipment downtime, while machine learning models are subsequently developed to as-sess whether those patterns are learnable and generalizable to unseen failure events. This dual approach enables a more complete characterization of industrial failure dynamics than either method alone.