[Reconstruction] EMD Inspection

[petteriTeikari]

Good quality recordings

1095

1104

Artifact removal working

1103

1116

4092

4095

Even when the constriction amplitude is small with high noise

4098

And with funky baseline

4102

Nice handling of missing chunks as well

4121

4135

Even with an artifact spike

4159

But then of course some traces are Problematic: Artifact spikes still left in the signal (after removing noise in IMF1-4):

1096

1146

4077

4080

4139

4143

4148

4156

4168

Questionable Seems like a reconstriction or is it just an artifact?

2091

[petteriTeikari]

After running the EMD the second time after further cleaning some of the spikes and imputing the data with missForest that seems to produce better imputations than the previously used fast StructTS Kalman filter from imputeTS, the denoising quality starts to seem really good:

1005

1010

Denoising performs rather well under heavy noise as well

4168

4211

And in some cases, one can inspect whether too much was taken from the extremities (in this case from max constriction), or was the denoising reasonable in the end:

1085

4137

In some cases, the small outlier "constrictions" could be constrictions, or just most likely outliers, but these were kept as they looked sort of "physiological", but could be removed later if wanted

4014

4046

4084

4114

missForest imputation with CEEMD handles better longer missing chunks than the simple Kalman filtering based

4121, missForest + CEEMD

4121, Kalman + CEEMD

For two cases, the missForest left some outlier spikes but they were removed on second run of missForest by manually marking those spikes as NAs and keeping the imputed values from previous runs.

See the difference for file 2056

before

after

And for file 4148

before

after

And the oscillations themselves might be a bit biometric feature as well?

Some subject show stronger oscillations (hiFreq)
from blue stimulus:

1149

Some from red stimulus

1172

And some people seem to have more or less constant oscillations throughtout the recording

And some people have rather identical oscillations on light onsets:

2049

2050

And one person had very pronounced light-dependent oscillations

And as technical notes, when the decomposition is "perfect" with no residue left, the last IMF becomes a noise component as well:

1064

And sometimes you get three "base IMFs" as typically I only have selected the residue and the last IMF to the "base"