Skip to content

Commit 5465b80

Browse files
authored
Keep last 3 benchmark shapes for fp8_quant and causal_conv1d (#129)
* Keep last 3 benchmark shapes for fp8_quant_py instead of first 3 The larger shapes (higher num_tokens and hidden_dim) are more representative of production workloads for FP8 quantization. * Keep last 3 benchmark shapes for causal_conv1d_py instead of first 3 The larger shapes (higher D and S) are more representative of production workloads for causal conv1d.
1 parent d773d99 commit 5465b80

2 files changed

Lines changed: 4 additions & 4 deletions

File tree

problems/helion/causal_conv1d_py/task.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -39,9 +39,9 @@ tests:
3939
- {"B": 4, "D": 64, "S": 128, "W": 4, "seed": 9173}
4040

4141
benchmarks:
42-
- {"B": 1, "D": 768, "S": 512, "W": 4, "seed": 31232}
43-
- {"B": 1, "D": 768, "S": 2048, "W": 4, "seed": 4052}
4442
- {"B": 1, "D": 1536, "S": 2048, "W": 4, "seed": 2146}
43+
- {"B": 1, "D": 2560, "S": 2048, "W": 4, "seed": 3129}
44+
- {"B": 1, "D": 2560, "S": 4096, "W": 4, "seed": 54352}
4545

4646
test_timeout: 180
4747
benchmark_timeout: 180

problems/helion/fp8_quant_py/task.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -46,9 +46,9 @@ tests:
4646
- {"num_tokens": 8, "hidden_dim": 4096, "group_size": 128, "seed": 9173}
4747

4848
benchmarks:
49-
- {"num_tokens": 1, "hidden_dim": 4096, "group_size": 128, "seed": 31232}
50-
- {"num_tokens": 16, "hidden_dim": 4096, "group_size": 128, "seed": 4052}
5149
- {"num_tokens": 256, "hidden_dim": 4096, "group_size": 128, "seed": 2146}
50+
- {"num_tokens": 256, "hidden_dim": 8192, "group_size": 128, "seed": 3129}
51+
- {"num_tokens": 4096, "hidden_dim": 7168, "group_size": 128, "seed": 54352}
5252

5353
test_timeout: 180
5454
benchmark_timeout: 180

0 commit comments

Comments
 (0)