Model phi performance fix

[slokesha]

What does this PR do?

This PR fixes the performance drop for the Phi model
Added graph break with mark_step for lazy mode and attn_softmax_bf16 flag, which also improves performance in some cases.

HL-SMI Version: hl-1.22.0-rc-fw-
Driver Version: 1.22.0-48ef525
Nic Driver Version: 1.22.0-48ef525

image: artifactory-kfs.habana-labs.com/docker-local/1.22.0/ubuntu22.04/habanalabs/pytorch-installer-2.7.1:1.22.0-543

[karol-brejna-i]

@slokesha The PR looks good. Can you share some info on how to validate (like: which test to run).

[12010486]

Chiming in.
Tests that were run, and I believe you can use to check, are:

cd optimum-habana/
pip install -e .
python -m pip install .[tests]
pip install pytest
PT_HPU_LAZY_MODE=1 python -m pytest tests/test_text_generation_example.py --device gaudi3 -v -s --token=xxx --junitxml=/tmp/test_run_microsoft_phi.xml --log-cli-level 20 -k test_text_generation_bf16_1x[microsoft/phi-2-1-False-False]

and

PT_HPU_LAZY_MODE=1 python -m pytest tests/test_text_generation_example.py --device gaudi2 -v -s --token=xxx --junitxml=/tmp/test_run_microsoft_phi.xml --log-cli-level 20 -k test_text_generation_fp8[microsoft/phi-2-1-1-True-128-128] 

[regisss]

Not sure we should add this to the generation config as there could be other ways of running the model in fp8. Maybe we could simply check the value of the env variable QUANT_CONFIG in the modeling file since this is how args.quant_config is set up:

optimum-habana/examples/text-generation/run_generation.py

Line 457 in 2ac859e

args.quant_config = os.getenv("QUANT_CONFIG", "")

WDYT?

[karol-brejna-i]

@slokesha Are you still working on this PR?