Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Load .xdf files with gaps correctly #467

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

bkloeckl
Copy link
Contributor

Fix #385. Correctly load xdf files with incomplete data/gaps by inserting nan for missing data instead of interpolation.

marker_ids=None,
prefix_markers=False,
fs_new=None,
gap_threshold=1.0,
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a docstring entry please?

@cbrnr
Copy link
Owner

cbrnr commented Jan 15, 2025

There seems to be something wrong when resampling the data, but the problem also exists on main. When loading the file sub-13_ses-S001_task-HCT_run-001_eeg.xdf, the signal scaling (at least for the EEG channels) changes as well as the location of the gap:

Using the default of 52 Hz:
52Hz

Using 256 Hz:
256Hz

It looks like the first one shows the correct signals. I'm not sure what the problem is, because the streams are always resampled (this is necessary when loading 2 or more streams). @DominiqueMakowski maybe you have time to look?

@DominiqueMakowski
Copy link

DominiqueMakowski commented Jan 15, 2025

I've tested with NK and I it works "as expected"

import neurokit2 as nk

df, sr = nk.read_xdf("sub-13_ses-S001_task-HCT_run-001_eeg.xdf", upsample=0.25)
df[["TP9", "TP10", "LUX", "ECGBIT1"]].iloc[0:60000].plot(subplots=True)

image

df, sr = nk.read_xdf("sub-13_ses-S001_task-HCT_run-001_eeg.xdf", upsample=0.5)
df[["TP9", "TP10", "LUX", "ECGBIT1"]].iloc[0:120000].plot(subplots=True)

image

The scaling issue is quite odd, but probably related to some internal handling

@cbrnr
Copy link
Owner

cbrnr commented Jan 15, 2025

Wait, but here you're using your own (=NK) implementation, right? You did not compare it to what MNELAB is doing in this PR?

@DominiqueMakowski
Copy link

well no but to be honest I don't really have a good grasp on what mnelab does for xdf 😕

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

read_raw_xdf(): loading assumes evenly sampling
3 participants