Power up your data with signal processing; a use case at the Intensive Care Unit

By: Mattia Fornasa (Pacmed)

If you’re anything like me, you’ve learned about random forests by classifying the three species of the iris flower from neat tables of petal sizes. Or maybe your first regression algorithm predicted house prices using structured data about their number of rooms and floor area.

Unfortunately, reality is not always as nice and machine-learning problems rarely come with such well-defined and pre-packaged data sets.

Feature engineering and signal processing

Most of the times your original inputs need some clever massaging before being fed to the model. For example, in a classification problem, it is good practice to start by identifying the quantities that better distinguish one class from the other. Even if feature engineering can be quite challenging.

Coming up with features is difficult, time-consuming, requires expert domain knowledge. When working applications of learning, we spend a lot of time tuning the features. — Prof. Andrew Ng

Also, your initial inputs may be what we call signals or, broadly speaking, any stream of values recorded and outputted by a device. Think of a voice message as an example of 1D signal, or a selfie in the case of 2D. Whether you want to recognise the nationality of the person speaking or classify facial expressions, you first need to transform your signal into meaningful features that a model can understand and use.

Deciding how to process your signals is not always obvious. It normally requires a deep understanding of the overall problem, a good knowledge of the underlying distributions of your data and a basic familiarity with how the data are recorded. Ah… and a fair amount of creativity doesn’t hurt.

Strictly speaking, feature engineering and signal processing are different tasks. However, they share the notion that it doesn’t matter how sophisticated an algorithm is, the performance of your model will always be suboptimal if you start from uninformative features.

At the end of the day, some machine-learning projects succeed and some fail. What makes the difference? Easily the most important factor is the features used. — Prof. Pedro Domingos

The article you’re reading discusses how, at Pacmed, we implemented signal processing in the context of hospital intensive care. This example showcases the benefit of investing time and manpower in carefully and smartly preparing your features before rushing to model training.

Predicting readmission to the Intensive Care Unit

At Pacmed, we build machine learning products that help doctors and nurses make decisions. The tool we are developing with Dr. Patrick Thoral, MD and Dr. Paul Elbers, MD, PhD at the Intensive Care Unit of AmsterdamUMC (location VUmc) helps intensive care doctors determine when it is safe to discharge their patients. On average, 5.3% of the patients that are discharged from the ICU at the AmsterdamUMC either come back or pass away within 7 days after discharge. These adverse outcomes may be prevented by allowing the patient to remain at the ICU a few more days. Using the data gathered during the patient’s stay at the ICU, we can build a classification model that predicts the patient’s probability of being readmitted or passing away, if he/she were to be discharged today. Providing this information to the intensive care doctors can improve their decision making process when assessing whether to keep the patient or discharge him/her to the ward. You can read more about this project in these two blog articles.

Thanks to the abundance of data gathered daily, decisions based on machine-learning algorithms can improve the quality of the care provided in the Intensive Care Unit of a hospital (Source: iStockphoto).

The ICU is a very rich place in data since, most of the time, patients are connected to machines that record information with a frequency as high as once per minute. Apart from vital signals (as heart rate, temperature and blood pressure), our database contains also the results of performed lab tests and administered medicines. As an example, figure 1 shows the values of heart rate recorded from one typical patient. We cannot feed this stream of numbers to our classifier: informative and understandable features need to be extracted from the plot. But which ones and how?

Figure 1: All the values of the heart rate (in beats per minute) recorded from a patient during his/her admission at the ICU of Amsterdam UMC (location VUmc).

Signal processing and feature engineering at the ICU

We started by binning the data into 24-hours time windows (see the vertical black lines in figure 1). Then our decisions were motivated by three ideas:

1. We want to capture short-term (day-level) effects in the signal, namely, its average value, how much it fluctuates, possible temporal trends and how frequently it is measured. This gives rise to 8 features per day (see figure 2). An accurate day-level description of the patient is particularly important in the first day of the admission (in order to assess the severity of his/her condition at arrival) and on the last day (when we decide whether to discharge him/her).
2. We also want to model long-term effects, in order to pick up major trends in the patient’s well-being. This generates 11 admission-level features, similar to day-level ones but computed now over the whole admission.
3. A signal that deviates significantly from that of the other patients in the cohort can be an indication of a problematic condition. Therefore, for each patient we subtract from each signal the average of the same signal among all the patients. Large values of this quantity characterise unusual behaviours. 13 features are built from this, similar to those in points 1 and 2.

Figure 2: For each of the 24-hours time window of the heart rate signal in figure 1, we computed 8 features: its average value (solid red line), its standard deviations (pink filled area), the lowest, highest, first and last values (4 red dots), the slope of a linear fit to the data (dashed red line) and the number of values available in a day.

Following these 3 principles, we ended up with 32 features for each signal. In reality, the number increased further since we noticed that some variables show a non-linear dependency on the outcome. Therefore we decided to engineer new features to capture these non-linearities, among other effects. At this point we were overflowed with features: each patient was described by a set of approximately 5000 quantities, many of them correlated with each other. It was necessary to filter such an extensive pool of features, in order to reduce variance.

We decided to use (Logistic) LASSO, whose regularisation term helps put to zero the coefficients of the less informative features. Automatic feature selection is a task that showcases nicely the added value of machine learning: it may be intuitive for an intensive care doctor that high blood pressure in a patient the day of his/her discharge may be an indication of high readmission probability, but what is more relevant? An on-average high value or the highest point recorded that day? Or is it the difference between the values in the first and last day?

Results

After feature selection we ended up with ~200 features per patients: we were then ready to train a classifier. We tested several algorithms and reached the best performance with a Support-Vector-Machine (SVM) algorithm delivering an area under the ROC curve of 0.833. I have browsed the literature available online to compare with similar algorithms and I couldn’t find a performance higher than ours. Note that with unbalanced data sets (like in this case), the area under the ROC curve may not be the most relevant metric to evaluate the performance of a model. A useful alternative is the area under the Precision-Recall curve.

We were also very happy when we noticed that a naïve L2 logistic regression performs equally as well as the more sophisticated SVM. We interpreted this as a confirmation that it was a good idea to invest time in setting up our features carefully: we managed to make linear models sensitive to non-linear effects, enhancing their predictive power to the level of more complex classifiers.

For us at Pacmed, this is not just a nice accessory but a crucial achievement: to fully harvest the value of AI, synergy between doctor and technology is essential. Therefore, our models must be clear and explainable, not only accurate. That is why we normally prefer more transparent algorithms (as logistic regression) over more complex ones. For our readmission model, due to extensive signal processing and feature engineering, this does not come with a compromise in performance.

Discussion

There are no standard packages for signal processing since the solution needs to be tuned to the specific problem at hand. This is why the task is akin to craftsmanship. However, there are techniques that proved to be quite effective for specific problems, for example convolution filters for image classification or Fourier analysis for audio processing and filtering.

Some may also argue that neural networks invalidated the need for feature engineering since it’s up to the network itself to identify and build the most informative combination of features. On the other hand, this strategy works only if it is accompanied by a large data set to learn from.

Conclusion

It’s quite a challenge for doctors working at the ICU to gain intuition on the future well-being of a patient simply by staring at raw unstructured signal data. That is why having a smart signal processor can, to a good effect, complement medical expertise by rearranging the information available in the data and isolating the most precious bits.