[Reland] Attention(23) CUDA

[titaiwangms]

Reland reason

Reland #26466

The previous PR was reverted because it fails on the test:

1. Windows GPU CUDA CI Pipeline Test Job
2. Windows GPU TensorRT CI Pipeline Test Job

This PR includes the correct fix.

---

Description

This pull request introduces significant improvements and expanded support for multi-head attention kernels in ONNX Runtime, particularly focusing on supporting both 3D (BSNH) and 4D (BNSH) QKV input formats. The changes enhance flexibility, correctness, and maintainability for attention operations across CPU and CUDA implementations.

Expanded QKV Input Format Support

• Added support for 4D QKV input format (Q_K_V_BNSH) in CUDA attention kernels, including proper handling for both cases with and without past/present states, and enforcing that bias is not supported for this format. This includes logic to avoid unnecessary transposes and to write outputs directly when possible. [1] [2] [3] [4] [5] [6] [7]

Kernel and Operator Documentation Updates

• Updated OperatorKernels.md to document the new Attention operator inputs and outputs for both 3D and 4D formats, specifying supported tensor types for each input.

Correctness and Consistency Fixes

• Fixed the computation of causal attention indices in CUDA softmax kernels by clarifying and correcting the offset calculation for causal masking. [1] [2] [3] [4]
• Updated workspace allocation logic for QKV preparation to ensure correct workspace usage for new formats.

Attention Parameter and Helper Refactoring

• Added is_output_bnsh field to AttentionParameters to indicate output format and updated logic to use this for output placement and transposition decisions. [1] [2]
• Refactored CPU attention implementation to use the new attention_helper namespace for output mode enums and output shape computation, improving code clarity and maintainability. [1] [2] [3]

Minor Cleanups

• Removed outdated asserts and improved debug output strings for QKV preparation functions to clarify format and state handling. [1] [2] [3]

These changes collectively improve the flexibility, correctness, and maintainability of attention kernel implementations in ONNX Runtime, especially for advanced transformer models and large language model workloads.

NOT supported in this PR

• Boolean mask
• GQA
• Softcap
• Softmax precision
• qk_output_mode other than -1 and 0