Add challenge 83: Flash Attention (Hard)

[claude]

Summary

• Adds challenge 83: Flash Attention (Hard difficulty)
• Implements tiled causal multi-head self-attention using the online softmax algorithm
• Q, K, V tensors of shape (num_heads, seq_len, head_dim), causal mask applied
• Requires tiled computation — full seq_len × seq_len matrix must never be materialized
• Performance test: num_heads=8, seq_len=4096, head_dim=64

What makes this distinct

• Existing challenges (53, 12, 6, 80, etc.) all compute attention but allocate the full N×N score matrix
• This challenge specifically targets the Flash Attention algorithm: tiling over K/V blocks + online softmax recurrence to compute exact attention without the quadratic memory allocation
• Teaches the running-max / running-normalizer trick that is foundational to all modern long-context LLM inference

Test plan

• challenge.py imports and runs locally
• All functional tests pass (--action submit on NVIDIA TESLA T4)
• pre-commit run --all-files passes (black, isort, flake8, clang-format)
• All 6 starter files present and compile/run without errors
• Solution file not included in PR

🤖 Generated with Claude Code

[shxjames]

[shxjames]

[shxjames]

@kunal-mansukhani