Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add run_chunk() to deal with tile format dataset #44

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

add run_chunk() to deal with tile format dataset #44

wants to merge 1 commit into from

Conversation

Michaelsqj
Copy link
Contributor

if the dataset is tile format, it will update and gets trained in run_chunk()

if the dataset is tile format, it will update and gets trained in run_chunk()
@zudi-lin zudi-lin linked an issue Jun 17, 2020 that may be closed by this pull request
@zudi-lin
Copy link
Owner

Thanks for the pull request! Could you please help solve the following problems:

  1. Make the dataloader iterable. When run_chunk() is called, there is no iter(self.dataloader), which means next(self.dataloader) doesn't work at the train() function. You may add iter(self.dataloader) to run_chunk() or make dataloader an iterator in build_dataloader().
  2. Use self.total_num_iters in trainer. The config options (cfg) is not mutable in the scripts (see this line: https://github.com/zudi-lin/pytorch_connectomics/blob/master/scripts/main.py#L41). Therefore self.cfg.SOLVER.ITERATION_TOTAL = self.cfg.DATASET.DATA_CHUNK_ITER is not valid. Maybe add a self.total_num_iters to the Trainer? Then train() and run_chunk() need to be changed accordingly.

@zudi-lin zudi-lin removed a link to an issue Jun 27, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants