Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Intan loader #76

Open
wants to merge 34 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
15f4086
working example
tdincer Feb 2, 2023
47befa3
code cleaning
tdincer Feb 3, 2023
1c1b169
add time unit
tdincer Feb 3, 2023
5b41b0a
read data based on the data type manual
tdincer Feb 3, 2023
c3f4540
return header as well
tdincer Feb 3, 2023
b7d1d70
read other files as well
tdincer Feb 3, 2023
1e1abae
Merge pull request #73 from tdincer/intan
JaerongA Feb 3, 2023
f44416f
refactor: :art: streamline the code
JaerongA Feb 6, 2023
500ca38
Update docstring
JaerongA Feb 6, 2023
fc5839f
remove unnecessary print statements
JaerongA Feb 8, 2023
d8cff9b
apply regex to filter data files
JaerongA Feb 8, 2023
30960f5
Update element_interface/intan_loader.py
JaerongA Feb 8, 2023
0d251e9
dtype=np.uint16 for some signal types
JaerongA Feb 8, 2023
e8d31c4
revert regex filtering
JaerongA Feb 9, 2023
10f5e0e
suggestions from code review
JaerongA Feb 9, 2023
661b705
Update element_interface/intan_loader.py
JaerongA Feb 10, 2023
c41260d
add a comment for the conversion fix
JaerongA Feb 10, 2023
8b14781
Update element_interface/intan_loader.py
tdincer Feb 10, 2023
9f2221f
Update element_interface/intan_loader.py
JaerongA Feb 10, 2023
283aac3
allow reading only info and timestamps
JaerongA Feb 13, 2023
eb4d611
Merge pull request #75 from JaerongA/intan
tdincer Feb 13, 2023
575b8e9
fix empty file_path
tdincer Feb 13, 2023
d1c9c1c
version bump
tdincer Feb 13, 2023
081ed02
remove the condition before if
tdincer Feb 13, 2023
449712a
add source reference
tdincer Feb 13, 2023
3d056f2
single file loader
tdincer Feb 14, 2023
1f99827
add code source
tdincer Feb 15, 2023
1fe2997
Update element_interface/intanloader/rhsutilities.py
tdincer Feb 15, 2023
ec59cff
output header
JaerongA Feb 15, 2023
947d27b
change import
tdincer Feb 16, 2023
50caa0e
Merge branch 'intan' of https://github.com/tdincer/element-interface …
tdincer Feb 16, 2023
00c9ac7
remove module
tdincer Feb 16, 2023
e2819d6
blackify
tdincer Feb 16, 2023
791ef09
Merge branch 'main' into intan
dimitri-yatsenko May 25, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@
Observes [Semantic Versioning](https://semver.org/spec/v2.0.0.html) standard and
[Keep a Changelog](https://keepachangelog.com/en/1.0.0/) convention.

## [0.6.0] - 2023-02-13

+ Add - Intan loader

## [0.5.4] - 2023-05-25

+ Fix - DANDI URL for uploads where `staging=False`.
Expand Down Expand Up @@ -71,6 +75,8 @@ Observes [Semantic Versioning](https://semver.org/spec/v2.0.0.html) standard and

+ Add - Readers for: `ScanImage`, `Suite2p`, `CaImAn`.


[0.6.0]: https://github.com/datajoint/element-interface/releases/tag/0.6.0
[0.5.4]: https://github.com/datajoint/element-interface/releases/tag/0.5.4
[0.5.3]: https://github.com/datajoint/element-interface/releases/tag/0.5.3
[0.5.2]: https://github.com/datajoint/element-interface/releases/tag/0.5.2
Expand Down
85 changes: 85 additions & 0 deletions element_interface/rhs_perchannel_loader.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
import numpy as np
from pathlib import Path
from intanrhsreader import read_header


def load_rhs(folder: str, file_expr: str = "*"):
"""Load rhs data

Data type and coversions are based on https://intantech.com/files/Intan_RHS2000_data_file_formats.pdf.

Example:
# Read data
>>> rhs_data = load_rhs("/home/inbox/organoids21/032520_US_885kHz_sham", file_expr="amp*dat")

# Plot data
>>> import matplotlib.pyplot as plt
>>> plt.plot(rhs_data["time"], rhs_data["recordings"]["amp-B-000.dat"])
>>> plt.xlabel("Time (s)")
>>> plt.ylabel("Reading")
>>> plt.show()

Args:
folder (str): Folder that contains info.rhs, time.dat, and *.dat files
file_expr (str): pattern matching of file names to be read. Defaults to "*" (read all files).

Returns:
rhs_data (dict): RHS data.
rhs_data["header"] (dict): Header.
rhs_data["recordings"] (dict): Readings from various files
rhs_data["timestamps"] (np.array_like): Relative timestamps in seconds.
"""

rhs_data = {}

# Get header
header_filepath = next(Path(folder).glob("info.rhs"))
with open(header_filepath, "rb") as fid:
rhs_data["header"] = read_header(fid)

# Get timestamps
time_file = next(Path(folder).glob("time.dat"))

rhs_data["timestamps"] = (
np.memmap(time_file, dtype=np.int32)
/ rhs_data["header"]["frequency_parameters"]["amplifier_sample_rate"]
)

# Get data files
file_paths = Path(folder).glob(file_expr)

exclude_list = ["time", "info", "Zone.Identifier"]

file_paths = [
file
for file in file_paths
if not any(string in file.as_posix() for string in exclude_list)
]

# Get recording data
rhs_data["recordings"] = {}

for file_path in sorted(file_paths):
signal_type = file_path.stem.split("-")[0]

if signal_type == "amp":
signal = np.memmap(file_path, dtype=np.int16)
signal = signal * 0.195 # Convert to microvolts

elif signal_type == "board":
signal = np.memmap(file_path, dtype=np.uint16)
signal = (signal - 32768) * 0.0003125 # Convert to volts
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fudge factors. Please turn into named constants.


elif signal_type == "dc":
signal = np.memmap(file_path, dtype=np.uint16)
signal = (signal - 512) * 19.23 # Convert to milivolts

elif signal_type == "stim":
signal = np.memmap(file_path, dtype=np.uint16)
# convert the signal from 9-bit one's complement to standard encoding
current = np.bitwise_and(signal, 255) * rhs_data["header"]["stim_step_size"]
sign = 1 - np.bitwise_and(signal, 256) // 128
signal = current * sign

rhs_data["recordings"][file_path.stem] = signal
return rhs_data
2 changes: 1 addition & 1 deletion element_interface/version.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
"""Package metadata"""

__version__ = "0.5.4"
__version__ = "0.6.0"