Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MedShapeNet and example #9823

Open
wants to merge 11 commits into
base: master
Choose a base branch
from
Open

Conversation

jdhenaos
Copy link

@jdhenaos jdhenaos commented Dec 5, 2024

I have added MedShapeNet dataset, adding eight classes.

The next folders were ignored from MedShapeNet Dataset due to a lack of samples:

ASOCA (n=43)
AVT (n=45)
AutoImplantCraniotomy (n=14)
FaceVR (n=14)

This Dataset requires a size number as a parameter to download the same number of samples per shape to ensure class balance.

MedShapeNet is not divided by train and test by default. Therefore, train data is downloaded from the first n required number of samples, and the test data is downloaded from the last n required number of samples.

dgcnn classification example was modified to use both ModelNet and MedShapeNet.

@jdhenaos jdhenaos requested a review from wsad1 as a code owner December 5, 2024 20:49
@puririshi98
Copy link
Contributor

“MedShapeNet is not divided by train and test by default. Therefore, train data is downloaded from the first n required number of samples, and the test data is downloaded from the last n required number of samples.”
does this mean it is not doing a random split?

i would recommend the data be split randomly if train/val/test is not predefined

@puririshi98
Copy link
Contributor

root@bldgs-dune-b00-mab-3:/workspace/pytorch_geometric# python3 examples/dgcnn_classification.py -h
Traceback (most recent call last):
  File "/workspace/pytorch_geometric/examples/dgcnn_classification.py", line 8, in <module>
    import torch_geometric.transforms as T
  File "/usr/local/lib/python3.12/dist-packages/torch_geometric/__init__.py", line 21, in <module>
    import torch_geometric.datasets
  File "/usr/local/lib/python3.12/dist-packages/torch_geometric/datasets/__init__.py", line 83, in <module>
    from .medshapenet import MedShapeNet
  File "/usr/local/lib/python3.12/dist-packages/torch_geometric/datasets/medshapenet.py", line 8, in <module>
    from MedShapeNet import MedShapeNet as msn
ModuleNotFoundError: No module named 'MedShapeNet'
root@bldgs-dune-b00-mab-3:/workspace/pytorch_geometric# pip install medshapenet
DEPRECATION: Loading egg at /usr/local/lib/python3.12/dist-packages/lightning_utilities-0.11.8-py3.12.egg is deprecated. pip 25.1 will enforce this behaviour change. A possible replacement is to use pip for package installation. Discussion can be found at https://github.com/pypa/pip/issues/12330
DEPRECATION: Loading egg at /usr/local/lib/python3.12/dist-packages/faiss-1.7.4-py3.12.egg is deprecated. pip 25.1 will enforce this behaviour change. A possible replacement is to use pip for package installation. Discussion can be found at https://github.com/pypa/pip/issues/12330
DEPRECATION: Loading egg at /usr/local/lib/python3.12/dist-packages/pyg_lib-0.4.0-py3.12-linux-aarch64.egg is deprecated. pip 25.1 will enforce this behaviour change. A possible replacement is to use pip for package installation. Discussion can be found at https://github.com/pypa/pip/issues/12330
DEPRECATION: Loading egg at /usr/local/lib/python3.12/dist-packages/looseversion-1.3.0-py3.12.egg is deprecated. pip 25.1 will enforce this behaviour change. A possible replacement is to use pip for package installation. Discussion can be found at https://github.com/pypa/pip/issues/12330
DEPRECATION: Loading egg at /usr/local/lib/python3.12/dist-packages/igraph-0.11.8-py3.12-linux-aarch64.egg is deprecated. pip 25.1 will enforce this behaviour change. A possible replacement is to use pip for package installation. Discussion can be found at https://github.com/pypa/pip/issues/12330
DEPRECATION: Loading egg at /usr/local/lib/python3.12/dist-packages/lightning_thunder-0.2.0.dev0-py3.12.egg is deprecated. pip 25.1 will enforce this behaviour change. A possible replacement is to use pip for package installation. Discussion can be found at https://github.com/pypa/pip/issues/12330
DEPRECATION: Loading egg at /usr/local/lib/python3.12/dist-packages/opt_einsum-3.4.0-py3.12.egg is deprecated. pip 25.1 will enforce this behaviour change. A possible replacement is to use pip for package installation. Discussion can be found at https://github.com/pypa/pip/issues/12330
DEPRECATION: Loading egg at /usr/local/lib/python3.12/dist-packages/texttable-1.7.0-py3.12.egg is deprecated. pip 25.1 will enforce this behaviour change. A possible replacement is to use pip for package installation. Discussion can be found at https://github.com/pypa/pip/issues/12330
DEPRECATION: Loading egg at /usr/local/lib/python3.12/dist-packages/nvfuser-0.2.13a0+0d33366-py3.12-linux-aarch64.egg is deprecated. pip 25.1 will enforce this behaviour change. A possible replacement is to use pip for package installation. Discussion can be found at https://github.com/pypa/pip/issues/12330
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting medshapenet
  Downloading MedShapeNet-0.1.25-py3-none-any.whl.metadata (5.8 kB)
Collecting minio>=7.2.8 (from medshapenet)
  Downloading minio-7.2.12-py3-none-any.whl.metadata (6.5 kB)
Requirement already satisfied: tqdm>=4.66.5 in /usr/local/lib/python3.12/dist-packages (from medshapenet) (4.67.0)
Collecting numpy<1.26.0,>=1.18.5 (from medshapenet)
  Downloading numpy-1.25.2.tar.gz (10.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.8/10.8 MB 90.6 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [33 lines of output]
      Traceback (most recent call last):
        File "/usr/local/lib/python3.12/dist-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/usr/local/lib/python3.12/dist-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/local/lib/python3.12/dist-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 112, in get_requires_for_build_wheel
          backend = _build_backend()
                    ^^^^^^^^^^^^^^^^
        File "/usr/local/lib/python3.12/dist-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 77, in _build_backend
          obj = import_module(mod_path)
                ^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/lib/python3.12/importlib/__init__.py", line 90, in import_module
          return _bootstrap._gcd_import(name[level:], package, level)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
        File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
        File "<frozen importlib._bootstrap>", line 1310, in _find_and_load_unlocked
        File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
        File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
        File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
        File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
        File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
        File "<frozen importlib._bootstrap_external>", line 995, in exec_module
        File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
        File "/tmp/pip-build-env-51s0gzdh/overlay/local/lib/python3.12/dist-packages/setuptools/__init__.py", line 16, in <module>
          import setuptools.version
        File "/tmp/pip-build-env-51s0gzdh/overlay/local/lib/python3.12/dist-packages/setuptools/version.py", line 1, in <module>
          import pkg_resources
        File "/tmp/pip-build-env-51s0gzdh/overlay/local/lib/python3.12/dist-packages/pkg_resources/__init__.py", line 2172, in <module>
          register_finder(pkgutil.ImpImporter, find_on_path)
                          ^^^^^^^^^^^^^^^^^^^
      AttributeError: module 'pkgutil' has no attribute 'ImpImporter'. Did you mean: 'zipimporter'?
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

note that CI is also failing with this issue.
to fix CI see how the imports are done for webqsp: https://github.com/pyg-team/pytorch_geometric/blob/master/torch_geometric/datasets/web_qsp_dataset.py#L156-L157

however, I am having trouble installing medshapenet through pip to verify if your code actually works. Is that how you install it? could you provide your setup commands? based on this i will also advise wether those instructions should be included in your code somewhere (i will decide based on the complexity of the set up instructions)

Copy link

codecov bot commented Dec 10, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 86.31%. Comparing base (bd5ae45) to head (48180f0).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #9823      +/-   ##
==========================================
- Coverage   87.07%   86.31%   -0.76%     
==========================================
  Files         490      490              
  Lines       32285    32280       -5     
==========================================
- Hits        28112    27863     -249     
- Misses       4173     4417     +244     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@xnuohz
Copy link
Contributor

xnuohz commented Dec 22, 2024

Should it be pip install MedShapeNet?
https://github.com/GLARKI/MedShapeNet2.0

@jdhenaos
Copy link
Author

Indeed, pip instal MedShapeNet works.

The problem is that MedShapeNet uses a down-grade version of numpy. By default, it downloads version 1.25.2, which works well with Python 3.10 but raises an error when recent versions like 3.12 are used.

I'm not sure how to proceed with this conflict. Any hint you can give will be welcome.

@jdhenaos
Copy link
Author

@xnuohz and @puririshi98

I have talked with the MedShapeNet. Looks like they already solved the problem, I was able to run the Dataset on Python 3.10.12.

Please corroborate and let me know if there is some pending issue to solve

Copy link
Contributor

@xnuohz xnuohz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left some comments:)

Comment on lines +51 to +57
train_dataset = MedShapeNet(root=root, size=50, split="train",
pre_transform=pre_transform,
transform=transform, force_reload=False)
print('Loading test data')
test_dataset = MedShapeNet(root=root, size=50, split="test",
pre_transform=pre_transform,
transform=transform, force_reload=False)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can the class initialization parameter signature be consistent with other datasets? in this way it can be much simpler here
also please share the training log of this new added dataset

Comment on lines +120 to +121
train_data, val_data, test_data = random_split(
stl_files, [train_size, val_size, test_size])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we move random splits out of the dataset? imo, you need to split it in training stage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants