Feature Request: Support dual normalization modes to compute speedup ratios and standard normalization

[ConvolutedDog]

Problem

Currently, normalize_against uses agg_df["NormalizationValue"] / agg_df["AvgValue"] to compute speedup ratios, which works well for timing metrics. However, for standard normalization of general metrics (memory metrics, instruction metrics, etc.), the correct formula should be agg_df["AvgValue"] / agg_df["NormalizationValue"].

Current Behavior

nsight-python/nsight/transformation.py

Lines 146 to 147 in 64f60bf

	# Normalize the AvgValue by the values of the normalization annotation
	agg_df["AvgValue"] = agg_df["NormalizationValue"] / agg_df["AvgValue"]

[ConvolutedDog]

@bastianhagedorn @Alok-Joshi

In my opinion, adopting standard normalization consistently would likely be the most intuitive approach, it allows users to handle interpretation based on their specific metric semantics.

The fact is only users themselves know which direction is desirable for their particular use case. By using standard normalization (current / baseline) consistently, we provide mathematical consistency and clarity. This approach would eliminate the need for users to choose between different modes (like "speedup" vs. "standard"), reducing potential confusion.

The library would focus on providing consistent mathematical normalization, while leaving the semantic interpretation to users based on their metric's characteristics.

[bastianhagedorn]

I’m aligned with the suggestion to use regular normalization (current / baseline). For speedups we can still support the workflow by combining normalize_against with derive_metric to compute the reciprocal:

def speedup(metric, n):
    return 1 / metric

@nsight.analyze.kernel(normalize_against="some_kernel", derive_metric=speedup)
def benchmark_matmul(n: int) -> torch.Tensor:
    ...

What do you think @Alok-Joshi ?

[Alok-Joshi]

@bastianhagedorn Makes sense to me.

[Alok-Joshi]

@bastianhagedorn has a WIP change for this