[Discussion] Feature ideas looking at other language ecosystem

[Moelf]

The package looks great and I hope this goes far. I see that Oliver already has an issue open re. DensityInterface.

My main interest here is to see if we (high-energy physics) can reuse your package for our likelihood-based fitting:

• https://arxiv.org/abs/1007.1727
• https://cds.cern.ch/record/1456844/files/CERN-OPEN-2012-016.pdf

Essentially it's the same likelihood ratio method, and I have a fun webpage with toy problem and the 2D nuisance parameter/parameter of interest scan:

• https://jiling.web.cern.ch/jiling/public/LikelihoodRatio.html (see Conditional Likelihood and Likelihood Ratio)

Now, our field has produced this for Python and C++ (many in each), I will high light two:

• https://pyhf.readthedocs.io/en/v0.7.0/
• https://indico.cern.ch/event/822074/contributions/3471458/attachments/1865561/3067487/20190619_TRExFitter_AS.pdf

Where some of the valuable utilities I wish we can also have here:

• pull plot (i.e. best fitted value and range of each nuisance parameters)
• Correlation matrix for each nuisance parameter
• impact ranking plot where the impact on parameter of interest from each nuisance parameter pre/post fit is plotted

[Moelf]

The automation of what is highlighted in the Pluto notebook is made into a not-very-complete package by me:

• https://github.com/JuliaHEP/LiteHF.jl

but from what I see, the only "non-standard" thing in all of this is the relationship between the nuisance parameter and the term in likelihood associated with it in our field. We essentially take a three-point estimate (up, nominal, down) and interpolate between them, and our nominal parts of the likelihood is always Poisson (because we always reduce the problem to binn-ed fitting)

[DanielVandH]

Hi @Moelf, thanks for your interest!

I'm a bit confused on what you are suggesting here - probably because I'm too tired currently - but these links are quite interesting. Are you just suggesting that it might be nice to have those three features you mention at the end? I've not come across those types of features before, at least in the context of profile likelihoods, but I think the functions I have in the package could be easily used to implement them.

[Moelf]

yeah, and essentially if your energy/time budget allows, I'd like to work out how can we (Julia community) best re-using each other's stuff.

ideally I can convince you to have certain features (for example those mentioned above), that would allow LiteHF.jl to use this package as the basis maybe

[DanielVandH]

I see. I agree that being able to re-use other's code where possible would be a good idea.

Regarding these features, as I mentioned I've not heard of them previously so I'm not sure how easy they would be to implement, and I am a bit busy now. I'd be happy to help you with a PR for it, or if you want to provide clear definitions for those three features (i.e. how they are calculated, test cases) I can try and get to it eventually.

It looks like some of this stuff is a bit specific to HEP, though, in which case it might be more fitting to build on this in LiteHF.jl and I could support you with more basic functionality where needed. Maybe I'm wrong.