[Question] Inquiry Regarding EXEC-DRIVEN Mode in Scarab v2.0

[Hong-git007]

Dear Scarab development team,

We are a research lab in Korea working on CPU microarchitecture.

First of all, we would like to express our sincere appreciation for your efforts in developing and maintaining Scarab v2.0. Your work has been invaluable to our research.

We have been using Scarab v1.0 from the HPS group for our simulations, primarily in EXEC-DRIVEN mode rather than trace mode. With that version, we were able to successfully simulate all SPEC2017 benchmarks using the SimPoint and checkpointing methodology.

However, after transitioning to Scarab v2.0, we are encountering issues with some benchmarks when attempting to run simulations in EXEC-DRIVEN mode using the same methodology. (We understand that the UDP paper seems to have used trace-based simulation, but we would strongly prefer to continue using EXEC-DRIVEN mode.)

Specifically, we observe failures in certain benchmarks only when perfect_bp and perfect_target are disabled. When these options are enabled, all benchmarks complete without issues.

Upon further investigation, we found that the failing instructions are all CF_ICALL branches. For these branches, the simulator receives unrealistically small NPC (next PC) values such as 0x12 from the PIN tool for OFF_PATH instructions. These values then cause assertion failures in Scarab (e.g., LINE_ADDR == 0) during simulation.

We would like to ask:

Has Scarab v2.0 been designed with support for EXEC-DRIVEN mode in mind?

Is there any recommended solution or workaround for the issue described above?

Thank you in advance for your time and support.

Sincerely,
First-year M.S. student
Yonsei University, South Korea

[hlitz]

adding @Kiminje

Hi @Hong-git007
Thank you for your feedback! Scarab v2.0 has been designed with support for exec-driven mode but our test suite may not cover the issues you are seeing. We recently fixed a similar issue (bc01907), can you confirm that you are on HEAD?

Please provide a detailed bug description and show the parameters and command line to reproduce the issue. We will have a look at it.

Heiner

[Hong-git007]

Thank you for your response.

I have confirmed that we are using the HEAD version of Scarab.
We run the simulation using the following command:
python3 ./bin/scarab_batch.py ./bin/batch_w0_s2.py --run --progress
The attached batch_w0_s2.py file contains the configuration we use.

We also attach the modified bp.c file, although the changes are limited to debug messages and minor adjustments to penalty cycles. The issue we’re reporting occurred even before any modifications were made to this file.

The simulation halts with the following assertion failure:
/home/hong/project/scarab/src/icache_stage.c:546: ASSERT FAILED (P=0 O=40438 I=33120 C=49927): ic->line_addr

To assist with debugging, we are also attaching the scarab.stdout and scarab.stderr logs captured during the problematic benchmark run, with BP and BTB debug options enabled.

Please let us know if you need any additional information.

Sincerely,
Hong

scarab_issue.zip

[Hong-git007]

I have reviewed commit bc01907, and from what I understand, the issue we're encountering seems quite similar.

In my analysis, it appears that PIN is supplying an incorrect target address. After a branch misprediction and subsequent recovery, this incorrect target address is being written into the BTB. Later, when the same branch instruction is encountered again, the BTB provides this wrong target address, which then leads to an invalid access in the icache and ultimately triggers the assertion failure.

Even though we built Scarab with the fix from bc01907, our logs still show incorrect target addresses being supplied — in fact, values smaller than 0x40.

For example, from one of our debug logs:
/home/hong/project/scarab/src/bp/bp.c:816: DEBUG_BP (P=0 O=40319 I=33009 C=49635): BP: op_num:40396 off_path:1 cf_type:CF_ICALL addr:4cb86b p_npc:40 t_npc:0x19 btb_miss:0 mispred:0 misfetch:1 no_tar:0 dir1 pred1 offset 4cb86d target 19

As shown, target is 0x19, which clearly seems unintended.

[Kiminje]

Hi @Hong-git007, thank you for your feedback!
You'll see the new commit when it's merged.
Please let me know if the new commit doesn't fix your issue.

[hlitz]

please let us know if the latest patch fixes the issue and close if resolved