@cthoyt @mali-git Let's discuss the format here

I would add a \midrule before a new interaction model starts.

As far as I saw we already have a method which collects a dataframe containing the combination of best/avg/worst results for each model. Thus, we would need to create a multi-index ['model', 'rank_type'], and add for each interaction model the entry

# published_results is a dictionary of the loaded results for one dataset
# example
assert published_results[('trouillon2016', 'complex')] == {
    'rank_type': 'best',
    'mean_reciprocal_rank': 0.692,
    'hits_at_1': 0.599,
    'hits_at_3': 0.759,
    'hits_at_10': 0.840,
}

for reference_key, model_name, metrics in published_results.items():
    rank_type_key = metrics.pop('rank_type')[0]
    reference = r'\cite{' + str(reference_key) + '}^{' + rank_type_key + '}'
    for metric_name, value in metrics.items():
        df[(model_name, reference), metric_name] = metric_value

I am not sure what is the best way to get the JSON configs from pykeen master into this repository 😅

I am not sure what is the best way to get the JSON configs from pykeen master into this repository 😅

The old configs are contained in reproducibility/configs. We could replace those.

Done in #18

Thanks!