fig. 04Case study
Hydraulic Condition Monitoring
Condition monitoring of hydraulic systems in agricultural machinery: classifying valve health from multivariate sensor data — with an honest, leakage-free evaluation.
Dataset
- Cycles
- 2,205 load cycles
- Sensors
- 17 sensors
- Cycle length
- 60 s per cycle
- Target
- Valve condition (ordinal classes)
UCI — Condition Monitoring of Hydraulic Systems
The interesting part
Most versions of this study report ~98% accuracy. That number is an artifact of data leakage — and the whole point of this project is refusing to report it. Toggle the split and watch the metric tell the truth.
The split decides what your metrics mean
2,205 test-bench cycles
CV macro-F1
0.98
⚠ too good to be true — it is
A random split scatters near-identical test-bench cycles across train and test. The model recognizes its own training data — and the score is fiction.
From that honest 0.53 baseline, feature engineering and tuning brought the final model to 90.6% test accuracy — a number that means something.
Results — Gradient Boosting + SMOTE
- Test accuracy
- 90.6%
- on the honest, grouped split
- Macro-F1
- 0.873
- balanced across classes
- Ordinal MAE
- 0.094
- errors land next door, not far off
- QWK
- 0.965
- quadratic weighted kappa
- ECE
- 0.023
- well calibrated out of the box
- Brier score
- 0.152
- probability quality
End-to-end workflow
Feature engineering
Statistical features distilled from 17 raw sensor time series
Model comparison
7 classifiers benchmarked under identical, grouped CV
Class imbalance
SMOTE oversampling for the minority valve conditions
Tuning & calibration
Hyperparameter search, then calibration analysis (Brier, ECE)
Deployment demo
FastAPI service with a web UI for live predictions
Key learning: how you split decides what your metrics mean. Reporting the leakage-free numbers is the whole point of this study.
The accompanying academic paper (German, AI module) is company-related and not public. The published repository contains the full anonymized code and prototype.