Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Questions about WAP #19

Open
bilal2vec opened this issue Apr 1, 2019 · 6 comments
Open

Questions about WAP #19

bilal2vec opened this issue Apr 1, 2019 · 6 comments

Comments

@bilal2vec
Copy link

Hi,

I was reimplementing WAP and had some questions.

  1. Did you try using a resnet based encoder instead of using a vgg or densenet?
  2. Did you calculate the BLEU score as an additional metric?
  3. What are the train and val loss of the models after training?
  4. Looking at your data iterator code, it looks like you don't resize images but group together images with the same size into a batch. Is this correct?
  5. In issue Question about validation set #8 (comment), you wrote that the 2013 ground truth latex labels aren't normalized. How much did the exprate decrease when training and validating on un-normalized data?
  6. When training, do you use the ground truth label of the previous timestep as the input to the model at the current timestep (teacher forcing), or do you instead use the model's prediction at the previous timestep?

Thanks

@JianshuZhang
Copy link
Owner

sorry for the late.

  1. densenet is best for CROHM dataset, if your training set is big enough, maybe resnet is better.
  2. I didn't try BLEU.
  3. You can see the training log.
  4. Yes, I didn't resize images.
  5. If your training latex and test latex are mis-match, the exprate will decrease a lot.
  6. use the ground truth label of the previous timestep as the input

@bilal2vec
Copy link
Author

bilal2vec commented Apr 29, 2019

Hi, thanks for the response

I tried training another WAP implementation (https://github.com/menglin0320/wap-on-tensor) on the new 2019 crohme dataset (https://www.cs.rit.edu/~crohme2019/task.html) but couldn't get the model to generalize to the new out of distribution validation set. Did you also experience problems with your model generalizing to the 2013 test set? If so, how were you able to solve that problem?

Thanks

@JianshuZhang
Copy link
Owner

Recently, we also take part in the CROHME2019 competition. Our model can be tested on CROHME 2013, 2014, 2016 dataset. So, I didn't meet your problems.

@Zhang-O
Copy link

Zhang-O commented Aug 28, 2019

@JianshuZhang
1.densenet is best for CROHM dataset, if your training set is big enough, maybe resnet is better.
how many pics will satisfy the ' big enough '? 20k will ok?

@JianshuZhang
Copy link
Owner

JianshuZhang commented Aug 28, 2019 via email

@Zhang-O
Copy link

Zhang-O commented Aug 28, 2019

thank you for your reply. if I want to change the backbone to resnet, pics num should be more than 40k.Is my understanding right?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants