Batch 2: Transformer + Scheduler specs (CLARA-critical)

[gHashTag]

Task

Create 8 Transformer component and scheduler .tri specifications for CLARA TA1.

Why These 8 Specs

• Transformer components are core to HSLM architecture
• Schedulers required for training loops (cosine decay)
• LayerNorm critical for stable training

Deadline: April 6, 2026 (DARPA CLARA proposal)

Specs to Create

File	Description	Priority
specs/algo/multi_head_attn.tri	Multi-head attention with concat/projection	P0
specs/algo/positional_enc.tri	Sinusoidal position embeddings	P0
specs/algo/feed_forward.tri	FFN: Linear → ReLU → Linear	P0
specs/algo/layernorm.tri	Layer normalization	P0
specs/algo/adamw.tri	Adam with decoupled weight decay	P0
specs/algo/sgd_momentum.tri	SGD with Nesterov momentum	P0
specs/algo/scheduler.tri	Cosine decay scheduler	P0
specs/algo/residual.tri	Residual connection (Add & Norm)	P0

Template

name: {name}
version: "1.0.0"
module: algo.transformer.{name}
description: "{description}"

types:
  {Name}Config:
    fields: [...]

functions:
  forward:
    params: [...]
    returns: {return_type}
    formula: |
      # Mathematical formula

behaviors:
  - name: {behavior_name}
    description: "Implementation details"

Dependencies

• Blocked by: Batch 1: RNN/Sequence + Optimizer specs #489 (Batch 1 complete)
• Related: DARPA CLARA TA1 – tri build & demo pipeline before Apr 17 #486 (CLARA demo)

Acceptance

• 8 .tri files created
• tri gen produces stubs
• Sacred constants verified

[gHashTag]

✅ Batch 2 Complete — 8 Transformer + Scheduler specs

Transformer Components (5):

• multi_head_attn.tri — Multi-head attention with concat/projection
• positional_enc.tri — Sinusoidal position embeddings
• feed_forward.tri — FFN: Linear → ReLU → Linear
• residual.tri — Add & Norm pattern
• layernorm.tri — Layer normalization

Optimizers (2):

• adamw.tri — Adam with decoupled weight decay
• sgd_momentum.tri — SGD with Nesterov momentum

Scheduler (1):

• scheduler.tri — Cosine decay learning rate schedule

Commit: d681271392

Total Specs: 26 .tri files created

• Phase 1: 16 core specs
• Batch 1: 14 RNN/Sequence/Optimizer/Loss specs
• Batch 2: 8 Transformer/Scheduler specs

CLARA Impact: Full HSLM Transformer spec coverage:

• Multi-head attention ✅
• Position encoding ✅
• Feed-forward networks ✅
• Layer normalization ✅
• Residual connections ✅
• AdamW optimizer ✅
• Cosine scheduler ✅

[gHashTag]

All 8 specs exist and verified:

• multi_head_attn.tri (52 lines)
• positional_enc.tri (42 lines)
• feed_forward.tri (47 lines)
• layernorm.tri (40 lines)
• adamw.tri (53 lines)
• sgd_momentum.tri (48 lines)
• scheduler.tri (41 lines)
• residual.tri (32 lines)

Total: 355 lines of .tri specs ✅