Causal Conv1d kernel

[superposition]

Description

Causal 1D convolution for temporal sequence modeling.

Operation

# Causal conv: only looks at past
# x: (B, D, T) → (B, D_out, T)
output = causal_conv1d(x, weight, bias=None)

Implementation

• Left-padded convolution (no future leakage)
• Fused with optional activation (SiLU for Mamba)
• Support for depthwise separable variant
• Efficient for small kernel sizes (3, 4, 7)

Tests

• Output at time t only depends on inputs ≤ t
• Matches F.conv1d with proper padding
• Gradient flows correctly
• gradcheck passes
• Fused activation produces correct result
• Works with various kernel sizes

Used In

WaveNet, Mamba, S4, temporal world models, TCN