Added cumulative k-factor plot #96

FrancescoMerlotti · 2025-06-04T10:32:05Z

Plot of the cumulative SMEFT K-factor.

FrancescoMerlotti · 2025-06-04T10:34:33Z

to be tested with this runcard

meta:
  title: SMEFT / SM plot (SMEFT K-factor)
  keywords: [example]
  author: Francesco Merlotti

pdf:
  id: NNPDF40_nnlo_aS_01180
  label: NNPDF40_nnlo_aS_01180

theoryid: 270

use_cuts: internal

dataset_inputs:
  - {dataset: HLLHC_HMDY_CC_EL_FINAL, cfac: [QCD, EWK], contamination: EFT_LO}
  - {dataset: HLLHC_HMDY_NC_EL_FINAL, cfac: [QCD, EWK], contamination: EFT_LO}

contamination_parameters:
  name: "W"
  value: 0.00008
  linear_combination: 
    Olq3: -15.94

template_text: |
  {@bsm_sm_ratio@}

actions_:
  - report(main=true)

ElieHammou · 2025-06-24T09:56:12Z

I have an error when the codes tries to combine arrays of different sizes. I think the cuts are applied in the SM predictions but not in the BSM factor parsing.

I have added those print statement:

print("Computing BSM factor ratios for dataset:", dataset.name)
print("Size of BSM dict:", bsm_dict[dataset.name].shape)
print("Size of err:", pred.loc[:,1:].std(axis=1).shape)
print("Size of cv:", cv.shape)
std = bsm_dict[dataset.name] * pred.loc[:,1:].std(axis=1).to_numpy() / cv

and I get this output:

Computing BSM factor ratios for dataset: HLLHC_HMDY_CC_EL_FINAL
Size of BSM dict: (16,)
Size of err: (16,)
Size of cv: (16,)
Computing BSM factor ratios for dataset: HLLHC_HMDY_NC_EL_FINAL
Size of BSM dict: (12,)
Size of err: (12,)
Size of cv: (12,)
Computing BSM factor ratios for dataset: ATLASTTBARTOT7TEV
Size of BSM dict: (1,)
Size of err: (1,)
Size of cv: (1,)
Computing BSM factor ratios for dataset: ATLASTTBARTOT8TEV
Size of BSM dict: (1,)
Size of err: (1,)
Size of cv: (1,)
Computing BSM factor ratios for dataset: ATLAS_TTB_DIFF_8TEV_LJ_TTRAPNORM
Size of BSM dict: (5,)
Size of err: (4,)
Size of cv: (4,)

----

Traceback (most recent call last):
  File "/Users/eliehammou/miniconda3/envs/simunet_cheat_sheet/bin/validphys", line 33, in <module>
    sys.exit(load_entry_point('validphys', 'console_scripts', 'validphys')())
  File "/Users/eliehammou/Software/simunet_git/SIMUnet/validphys2/src/validphys/scripts/main.py", line 10, in main
    vp.main()
  File "/Users/eliehammou/Software/simunet_git/SIMUnet/validphys2/src/validphys/app.py", line 163, in main
    a.main()
  File "/Users/eliehammou/miniconda3/envs/simunet_cheat_sheet/lib/python3.9/site-packages/reportengine/app.py", line 421, in main
    self.run()
  File "/Users/eliehammou/Software/simunet_git/SIMUnet/validphys2/src/validphys/app.py", line 158, in run
    super().run()
  File "/Users/eliehammou/miniconda3/envs/simunet_cheat_sheet/lib/python3.9/site-packages/reportengine/app.py", line 406, in run
    rb.execute_sequential()
  File "/Users/eliehammou/miniconda3/envs/simunet_cheat_sheet/lib/python3.9/site-packages/reportengine/resourcebuilder.py", line 210, in execute_sequential
    result = self.get_result(callspec.function,
  File "/Users/eliehammou/miniconda3/envs/simunet_cheat_sheet/lib/python3.9/site-packages/reportengine/resourcebuilder.py", line 380, in get_result
    fres = function(**kwdict)
  File "/Users/eliehammou/miniconda3/envs/simunet_cheat_sheet/lib/python3.9/site-packages/reportengine/figure.py", line 143, in wrapper
    return list(func(*args, **kwargs))
  File "/Users/eliehammou/Software/simunet_git/SIMUnet/validphys2/src/validphys/simunet_analysis.py", line 2194, in bsm_sm_ratio
    std = bsm_dict[dataset.name] * pred.loc[:,1:].std(axis=1).to_numpy() / cv
ValueError: operands could not be broadcast together with shapes (5,) (4,)

ElieHammou · 2025-06-24T10:27:34Z

I checked and this error disappears if I manually remove all the cuts.

ElieHammou · 2025-06-24T10:29:18Z

I have another issue when a include a new FK table. I do not understand it.

Here is my dataset input:

  - {dataset: HLLHC_TTBAR_14TEV_LJ_DIF_MTTBAR, frac: 0.75, cfac: [QCD], contamination: EFT_LO, new_commondata: True}

Here is my error:

(simunet_cheat_sheet) ~/PBSP/Projects/Interpretation/vp-analysis/data_visualisation/ - (main) > validphys SMEFT_effect_runcard.yaml
[WARNING]: Output folder exists: /Users/eliehammou/PBSP/Projects/Interpretation/vp-analysis/data_visualisation/output Overwriting contents
[INFO]: All requirements processed and checked successfully. Executing actions.
[INFO]: Loading HLLHC_TTBAR_14TEV_LJ_DIF_MTTBAR

----

Traceback (most recent call last):
  File "/Users/eliehammou/miniconda3/envs/simunet_cheat_sheet/bin/validphys", line 33, in <module>
    sys.exit(load_entry_point('validphys', 'console_scripts', 'validphys')())
  File "/Users/eliehammou/Software/simunet_git/SIMUnet/validphys2/src/validphys/scripts/main.py", line 10, in main
    vp.main()
  File "/Users/eliehammou/Software/simunet_git/SIMUnet/validphys2/src/validphys/app.py", line 163, in main
    a.main()
  File "/Users/eliehammou/miniconda3/envs/simunet_cheat_sheet/lib/python3.9/site-packages/reportengine/app.py", line 421, in main
    self.run()
  File "/Users/eliehammou/Software/simunet_git/SIMUnet/validphys2/src/validphys/app.py", line 158, in run
    super().run()
  File "/Users/eliehammou/miniconda3/envs/simunet_cheat_sheet/lib/python3.9/site-packages/reportengine/app.py", line 406, in run
    rb.execute_sequential()
  File "/Users/eliehammou/miniconda3/envs/simunet_cheat_sheet/lib/python3.9/site-packages/reportengine/resourcebuilder.py", line 210, in execute_sequential
    result = self.get_result(callspec.function,
  File "/Users/eliehammou/miniconda3/envs/simunet_cheat_sheet/lib/python3.9/site-packages/reportengine/resourcebuilder.py", line 380, in get_result
    fres = function(**kwdict)
  File "/Users/eliehammou/miniconda3/envs/simunet_cheat_sheet/lib/python3.9/site-packages/reportengine/figure.py", line 143, in wrapper
    return list(func(*args, **kwargs))
  File "/Users/eliehammou/Software/simunet_git/SIMUnet/validphys2/src/validphys/simunet_analysis.py", line 2184, in bsm_sm_ratio
    table = kitable(data=dataset, info=info)
  File "/Users/eliehammou/Software/simunet_git/SIMUnet/validphys2/src/validphys/plotoptions/core.py", line 306, in kitable
    data = data.load()
  File "/Users/eliehammou/Software/simunet_git/SIMUnet/validphys2/src/validphys/core.py", line 516, in load
    fktable = p.load()
  File "/Users/eliehammou/Software/simunet_git/SIMUnet/validphys2/src/validphys/core.py", line 617, in load
    return FKTable(str(self.fkpath), [str(factor) for factor in self.cfactors])
  File "/Users/eliehammou/miniconda3/envs/simunet_cheat_sheet/lib/python3.9/site-packages/NNPDF/nnpdf.py", line 3042, in __init__
    _nnpdf.FKTable_swiginit(self, _nnpdf.new_FKTable(*args))
RuntimeError: [utils] error: Could not open (PosixPath('/Users/eliehammou/miniconda3/envs/simunet_cheat_sheet/share/NNPDF/data/theory_270/fastkernel/HLLHC_TTBAR_14TEV_LJ_DIF_MTTBAR.pineappl.lz4'),)

FrancescoMerlotti · 2025-06-24T12:22:01Z

I understand the issue, it is related to bit where I try to load the kinematic information from the fktable, and it fails in the new format

ElieHammou · 2025-06-24T13:02:59Z

As it stands it gives me the following error when I use use_cuts: internal

Dataset: ATLASTTBARTOT7TEV
Table shape: (1, 3)
Cuts: [0]
Dataset: ATLASTTBARTOT8TEV
Table shape: (1, 3)
Cuts: [0]
Dataset: ATLAS_TTB_DIFF_8TEV_LJ_TTRAPNORM
Table shape: (4, 3)
Cuts: [0 1 2 3]
Dataset: ATLAS_TOPDIFF_DILEPT_8TEV_TTMNORM
Table shape: (5, 3)
Cuts: [0 1 2 3 4]
Dataset: ATLAS_TTBAR_8TEV_LJETS_TOTAL
Table shape: (1, 3)
Cuts: [0]
Dataset: ATLAS_TTBAR_13TEV_TTMNORM
Table shape: (8, 3)
Cuts: [1 2 3 4 5 6 7 8]

----

Traceback (most recent call last):
  File "/Users/eliehammou/miniconda3/envs/simunet_cheat_sheet/bin/validphys", line 33, in <module>
    sys.exit(load_entry_point('validphys', 'console_scripts', 'validphys')())
  File "/Users/eliehammou/Software/simunet_git/SIMUnet/validphys2/src/validphys/scripts/main.py", line 10, in main
    vp.main()
  File "/Users/eliehammou/Software/simunet_git/SIMUnet/validphys2/src/validphys/app.py", line 163, in main
    a.main()
  File "/Users/eliehammou/miniconda3/envs/simunet_cheat_sheet/lib/python3.9/site-packages/reportengine/app.py", line 421, in main
    self.run()
  File "/Users/eliehammou/Software/simunet_git/SIMUnet/validphys2/src/validphys/app.py", line 158, in run
    super().run()
  File "/Users/eliehammou/miniconda3/envs/simunet_cheat_sheet/lib/python3.9/site-packages/reportengine/app.py", line 406, in run
    rb.execute_sequential()
  File "/Users/eliehammou/miniconda3/envs/simunet_cheat_sheet/lib/python3.9/site-packages/reportengine/resourcebuilder.py", line 210, in execute_sequential
    result = self.get_result(callspec.function,
  File "/Users/eliehammou/miniconda3/envs/simunet_cheat_sheet/lib/python3.9/site-packages/reportengine/resourcebuilder.py", line 380, in get_result
    fres = function(**kwdict)
  File "/Users/eliehammou/miniconda3/envs/simunet_cheat_sheet/lib/python3.9/site-packages/reportengine/figure.py", line 143, in wrapper
    return list(func(*args, **kwargs))
  File "/Users/eliehammou/Software/simunet_git/SIMUnet/validphys2/src/validphys/simunet_analysis.py", line 2189, in bsm_sm_ratio
    x = info.get_xcol(table=table)[cuts]
IndexError: index 8 is out of bounds for axis 0 with size 8

where I have implemented the following print statements:

    for dataset in data.datasets:
        # get cuts from dataset
        cuts = dataset.cuts.load()
        # initialise figure
        fig, ax = plt.subplots()
        # get info
        info = get_info(dataset)
        # get kin table & get x
        table = kitable(data=dataset, info=info)
        print("Dataset:", dataset.name)
        print("Table shape:", table.shape)
        print("Cuts:", cuts)
        x = info.get_xcol(table=table)[cuts]

ElieHammou · 2025-06-24T13:36:01Z

There is also an issue when you implement the contamination_parameters key in the runcard to be a list of dictionaries rather than a dictionary.

E.g., this input:

contamination_parameters:
  - name: "W"
    value: 0.00008
    linear_combination: 
      Olq3: -15.94
  - name: 'Y'
    value: 0.00000
    # value: 0.00015
    linear_combination: 
      Olq1: 1.51606
      Oed: -6.0606
      Oeu: 12.1394
      Olu: 6.0606
      Old: -3.0394
      Oqe: 3.0394
  - name: 'Z'
    value: 0.0004
    # value: 0.0004
    linear_combination:
      O81qq: -164.705
      O8ut:  -164.705
      O8dt:  -164.705
      O8qt:  -164.705
      O8qu:  -164.705
      O8qd:  -164.705

gives this error:

(simunet_cheat_sheet) ~/PBSP/Projects/Interpretation/vp-analysis/data_visualisation/ - (main) > validphys SMEFT_effect_runcard.yaml
[WARNING]: Output folder exists: /Users/eliehammou/PBSP/Projects/Interpretation/vp-analysis/data_visualisation/output Overwriting contents
[INFO]: All requirements processed and checked successfully. Executing actions.

----

Traceback (most recent call last):
  File "/Users/eliehammou/miniconda3/envs/simunet_cheat_sheet/bin/validphys", line 33, in <module>
    sys.exit(load_entry_point('validphys', 'console_scripts', 'validphys')())
  File "/Users/eliehammou/Software/simunet_git/SIMUnet/validphys2/src/validphys/scripts/main.py", line 10, in main
    vp.main()
  File "/Users/eliehammou/Software/simunet_git/SIMUnet/validphys2/src/validphys/app.py", line 163, in main
    a.main()
  File "/Users/eliehammou/miniconda3/envs/simunet_cheat_sheet/lib/python3.9/site-packages/reportengine/app.py", line 421, in main
    self.run()
  File "/Users/eliehammou/Software/simunet_git/SIMUnet/validphys2/src/validphys/app.py", line 158, in run
    super().run()
  File "/Users/eliehammou/miniconda3/envs/simunet_cheat_sheet/lib/python3.9/site-packages/reportengine/app.py", line 406, in run
    rb.execute_sequential()
  File "/Users/eliehammou/miniconda3/envs/simunet_cheat_sheet/lib/python3.9/site-packages/reportengine/resourcebuilder.py", line 210, in execute_sequential
    result = self.get_result(callspec.function,
  File "/Users/eliehammou/miniconda3/envs/simunet_cheat_sheet/lib/python3.9/site-packages/reportengine/resourcebuilder.py", line 380, in get_result
    fres = function(**kwdict)
  File "/Users/eliehammou/Software/simunet_git/SIMUnet/validphys2/src/validphys/simunet_analysis.py", line 2005, in load_datasets_contamination
    cont_name = contamination_parameters["name"]
TypeError: list indices must be integers or slices, not str

added cumulative k-factor plot

431bed1

FrancescoMerlotti requested a review from ElieHammou June 4, 2025 10:32

ElieHammou added 2 commits June 24, 2025 10:41

Replaced error by warning if SMEFT K-fac not implemented

6f13408

Removed wrong argument in pred function

529e915

update with proper cuts

f16407c

ElieHammou and others added 2 commits July 21, 2025 13:33

Added legend

45549f9

fixed bug for new theories in plotoptions

c4709dd

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Added cumulative k-factor plot #96

Added cumulative k-factor plot #96

Uh oh!

FrancescoMerlotti commented Jun 4, 2025

Uh oh!

FrancescoMerlotti commented Jun 4, 2025 •

edited

Loading

Uh oh!

ElieHammou commented Jun 24, 2025

Uh oh!

ElieHammou commented Jun 24, 2025

Uh oh!

ElieHammou commented Jun 24, 2025

Uh oh!

FrancescoMerlotti commented Jun 24, 2025

Uh oh!

ElieHammou commented Jun 24, 2025

Uh oh!

ElieHammou commented Jun 24, 2025

Uh oh!

Uh oh!

Added cumulative k-factor plot #96

Are you sure you want to change the base?

Added cumulative k-factor plot #96

Uh oh!

Conversation

FrancescoMerlotti commented Jun 4, 2025

Uh oh!

FrancescoMerlotti commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ElieHammou commented Jun 24, 2025

Uh oh!

ElieHammou commented Jun 24, 2025

Uh oh!

ElieHammou commented Jun 24, 2025

Uh oh!

FrancescoMerlotti commented Jun 24, 2025

Uh oh!

ElieHammou commented Jun 24, 2025

Uh oh!

ElieHammou commented Jun 24, 2025

Uh oh!

Uh oh!

FrancescoMerlotti commented Jun 4, 2025 •

edited

Loading