Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pints issue 783 plot histogram #11

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

pints issue 783 plot histogram #11

wants to merge 6 commits into from

Conversation

chonlei
Copy link
Member

@chonlei chonlei commented May 7, 2019

Fix and close pints-team/pints#783.

@chonlei
Copy link
Member Author

chonlei commented May 7, 2019

Results are in submodule res, so I put the results here first, rather than keep tracking res...

mcmc_banana_AdaptiveCovarianceMCMC_1-2019-05-07-17:10:24-2
mcmc_banana_AdaptiveCovarianceMCMC_1-2019-05-07-17:10:24-5

@chonlei
Copy link
Member Author

chonlei commented May 7, 2019

The left plot shows all the available commit history, and right plot shows the last 12 commits, as histogram in #607c8e. Excluding the last commit -- which we want to check visually.

Then the runs from the last commit are plotted in purple #0504aa; I did lollipop plot if the number of runs is small, if it has too many (different number for left and right, because the frequency count is very different), then I do histogram.

@chonlei
Copy link
Member Author

chonlei commented May 7, 2019

@MichaelClerx @martinjrobins See what you think and what else to add/change. Then I will do for the rest of the existing tests.

@MichaelClerx
Copy link
Member

Can you make the labels clearer? No idea what's going on now just from looking at those

@MichaelClerx
Copy link
Member

Also not 100% sure I like the lollipops. I know we said histogram, but do you think this would be nicer as three swarm plots (side by side)?

@MichaelClerx
Copy link
Member

MichaelClerx commented May 8, 2019

Would also be good to see a histogram for all the data, so a plot done before any analysis. You're going a step too far here already :-)

@chonlei
Copy link
Member Author

chonlei commented May 8, 2019

Would also be good to see a histogram for all the data

Left is all the data, or you mean even with the last commit as one histogram? I thought the idea here was to test the last commit, so we should look at the distribution without the last commit and compare our last commit's results against that distribution. And since we are testing the last commit, so we should never put that as part of the histogram (the old historical commits) -- correct me if I am wrong.

so a plot done before any analysis

No analysis has been done yet... I just took out the last commit's results!

Also not 100% sure I like the lollipops. I know we said histogram, but do you think this would be nicer as three swarm plots (side by side)?

The lollipops are the results of the last commit which we wanted to test/check. Because the first few runs of that new commit, we won't have enough data to plot a histogram, so I decided to plot each data point out! So it is more like a delta mass, if you like, which gives the lollipops. And what three swarm plots?

@chonlei
Copy link
Member Author

chonlei commented May 8, 2019

Can you make the labels clearer? No idea what's going on now just from looking at those

Updated the label a bit.

mcmc_banana_AdaptiveCovarianceMCMC_1-2019-05-07-17:56:47-2

@MichaelClerx
Copy link
Member

The first question is: what does the distribution of all test results look like. Once we have that, we can discuss how to compare old results to new in the group!

@MichaelClerx
Copy link
Member

Updated the label a bit.

The ones in the legend, I meant!

@chonlei
Copy link
Member Author

chonlei commented May 8, 2019

The first question is: what does the distribution of all test results look like. Once we have that, we can discuss how to compare old results to new in the group!

I guess, to see the distribution of all test results look like, just ignore the lollipops plot, they should not change the overall distribution, so I guess that would do for discussion, no?

The ones in the legend, I meant!

OK, sorry, didn't get you, I will change that.

@chonlei
Copy link
Member Author

chonlei commented May 8, 2019

Is the legend any better now?

mcmc_banana_AdaptiveCovarianceMCMC_1-2019-05-08-10:49:03-2

@MichaelClerx
Copy link
Member

Maybe we don't need the hashes, and it could be something like "All 123 commits", "Last 10 commits", "Last commit" or something

@chonlei
Copy link
Member Author

chonlei commented May 8, 2019

OK, how about this?

mcmc_banana_AdaptiveCovarianceMCMC_1-2019-05-08-11:19:04-2

@MichaelClerx
Copy link
Member

Better, thanks

@MichaelClerx
Copy link
Member

I still think it's hard to compare the lollipops with the histograms.
Could we get:

  • Left: Single histogram of everything
  • Right: Swarmplots of (1) everything, (2) recent commits, (3) last commit

?

@chonlei
Copy link
Member Author

chonlei commented May 8, 2019

Should I make another plot method for that? I still like this one (at least for now), and we could use either one later?

@MichaelClerx
Copy link
Member

What do you think @martinjrobins ?

@martinjrobins
Copy link
Member

Thanks Chon. This is looking good. I'm happy representing the last commit with individual samples, since there isn't much data. The tails of the lollipops are a bit distracting though. Can we try cuttin the tails and just put dots down near the bottom/top of the plot? And probably make them a much different colour as well to make them stand out (red?)

Copy link
Member

@MichaelClerx MichaelClerx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good, happy to merge once Martin's ideas are implemented:

Thanks Chon. This is looking good. I'm happy representing the last commit with individual samples, since there isn't much data. The tails of the lollipops are a bit distracting though. Can we try cuttin the tails and just put dots down near the bottom/top of the plot? And probably make them a much different colour as well to make them stand out (red?)

@MichaelClerx
Copy link
Member

@chonlei maybe leave this for now, as Ben has been doing cool stuff in R that we might start using!

@chonlei
Copy link
Member Author

chonlei commented May 17, 2019

@chonlei maybe leave this for now, as Ben has been doing cool stuff in R that we might start using!

Had a look at Ben's ones, I still think this slightly simpler histogram plot might be useful along with his plots!

@MichaelClerx
Copy link
Member

Me too, but I'm not sure how we're going to be making plots in the future. Might change to a system where all the plotting is done in R!
See pints-team/pints#832

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add plot of distribution of previous results
3 participants