Skip to content

[WIP] Specialize the two hot container shapes + add build-time benchmark#140

Closed
acostarelli wants to merge 2 commits into
mainfrom
ac/specialize-containers
Closed

[WIP] Specialize the two hot container shapes + add build-time benchmark#140
acostarelli wants to merge 2 commits into
mainfrom
ac/specialize-containers

Conversation

@acostarelli

Copy link
Copy Markdown
Member

Claude generated, I will review later.

Add concretely-typed container_spec / sparse_container_spec methods for the two
dominant axis shapes — dense (names::Vector{String}, time_steps::UnitRange{Int})
and sparse (names, segments, time_steps) — alongside the generic Vararg
fallbacks. Every add_*_container! reaches them automatically through the
@generated _add_container! splat, so no call sites change. The sparse
specializations hardcode the key tuple type, eliding the runtime
eltype(Iterators.product(axs...)) computation; the dense ones pin an inferable
return type. Other shapes fall through to the unchanged generic methods.

Add test/performance/container_build.jl, a dependency-free (@elapsed/@allocated)
benchmark of container build-time for these shapes, and wire it into the
existing main-vs-branch performance_test.yml so a PR comment compares the
specialized methods (branch) against the unaware Vararg baseline (main).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds targeted Julia method specializations to reduce build-time overhead for the two most common optimization-container shapes, and introduces a CI-run performance script to compare container allocation costs between main and this branch.

Changes:

  • Add specialized container_spec methods for names::Vector{String} × time_steps::UnitRange{Int} (including Float64 NaN-fill path).
  • Add specialized sparse_container_spec methods for names × time and names × segments × time by hardcoding hot key tuple types.
  • Add a new build-time benchmark script and extend the performance workflow to run it on both main and the PR branch, posting outputs to the PR.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
test/performance/container_build.jl New container build-time benchmark for dense/sparse “hot shapes” and wrapper paths.
src/utils/jump_utils.jl Adds concrete specializations for dense and sparse container construction to reduce overhead vs generic Vararg methods.
.github/workflows/performance_test.yml Runs the new container benchmark for main and branch and includes results in the PR comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +94 to +99
),
]
for (label, f) in cases
ns, bytes = bench(f)
@printf("%-28s %8d %14.1f %14d\n", label, n, ns, bytes)
end
Comment on lines +158 to +162
for (label, f) in
[("add_variable (dense)", dense), ("add_constraints (sparse)", sparse)]
ns, bytes = bench(f)
@printf("%-28s %8d %14.1f %14d\n", label, n, ns, bytes)
end
Comment on lines +145 to +147
() -> begin
empty!(container.constraints)
IOM.add_constraints_container!(
@github-actions

github-actions Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Performance Results
Main

Network: 10 nodes, 13 edges, 3 cost segments
Generators: 5, Demands: 5
Loss coefficients (a, b, c) per generator:
  n1: a=0.007287  b=0.005968  c=0.003850
  n2: a=0.001094  b=0.009002  c=0.007028
  n3: a=0.007897  b=0.008552  c=0.003001
  n4: a=0.008179  b=0.002530  c=0.009637
  n5: a=0.009778  b=0.009934  c=0.009099

Solver logs: /home/runner/work/InfrastructureOptimizationModels.jl/InfrastructureOptimizationModels.jl/main/test/performance/logs/solver_2026-07-24T18-18-57.log

==============================================================================================================================================
Bilinear Approximation Benchmarks
  Refinement = depth for all methods
==============================================================================================================================================
Method          R   Vars Constrs   Bins    Objective   Gap(%) MIPGap(%)     LowerBnd  rmse δbi   max δbi   rmse δq    max δq  build_t  solve_t
----------------------------------------------------------------------------------------------------------------------------------------------
NLP (Ipopt)     -     40     105      0     0.956760        -         -            -  0.00e+00  0.00e+00  0.00e+00  0.00e+00   0.0313   0.0043

NLP (Uno)       -     40     105      0     0.956760   0.0000         -            -  0.00e+00  0.00e+00  0.00e+00  0.00e+00   0.0012   0.0011

Bin2+sSOS       4    190     535      0     1.180700  23.4061    0.0000     1.180700  8.07e-02  1.41e-01  2.49e+01  5.48e+01   0.0051   1.7687
Bin2+sSOS       6    250     655      0     1.098175  14.7806    0.0000     1.098175  1.26e-01  2.86e-01  5.47e+00  9.07e+00   0.0055   7.3837
Bin2+sSOS       8    310     775      0     1.053069  10.0661    0.0088     1.052976  8.62e-02  2.43e-01  4.18e+01  9.32e+01   0.0055  26.2279

Bin2+mSOS       4    310     805    120     1.180700  23.4061    0.0000     1.180700  8.07e-02  1.41e-01  2.49e+01  5.48e+01   0.0023   1.7285
Bin2+mSOS       6    430    1045    180     1.098175  14.7806    0.0092     1.098074  1.26e-01  2.86e-01  5.47e+00  9.07e+00   0.0024   7.3728
Bin2+mSOS       8    550    1285    240     1.053069  10.0661    0.0045     1.053021  8.62e-02  2.43e-01  4.18e+01  9.32e+01   0.0025  17.6821

Bin2+Saw        4    310    1075    120     0.985823   3.0376    0.0000     0.985823  1.08e-01  2.49e-01  1.11e+02  2.44e+02   0.0031   5.7551
Bin2+Saw        6    430    1495    180     0.958677   0.2004    0.0077     0.958603  7.52e-02  1.68e-01  8.94e+01  1.71e+02   0.0036  13.1208
Bin2+Saw        8    550    1915    240     0.956996   0.0247    0.0072     0.956927  7.52e-02  1.68e-01  1.37e+02  2.09e+02   0.0043  49.5348

HybS+sSOS       4    310    1165      0     0.812261  15.1030    0.0000     0.812261  5.52e-01  1.00e+00  2.04e+02  2.97e+02   0.0066   2.6651
HybS+sSOS       6    410    1545      0     0.891208   6.8515    0.0000     0.891208  5.49e-01  1.00e+00  3.69e+02  7.09e+02   0.0072   4.8605
HybS+sSOS       8    510    1925      0     0.934789   2.2965    0.0087     0.934707  5.48e-01  1.00e+00  3.47e+02  5.22e+02   0.0075  15.4469

HybS+mSOS       4    390    1345     80     0.812261  15.1030    0.0046     0.812224  5.52e-01  1.00e+00  2.04e+02  2.97e+02   0.0033   2.8853
HybS+mSOS       6    530    1805    120     0.891208   6.8515    0.0000     0.891208  5.49e-01  1.00e+00  3.69e+02  7.09e+02   0.0037   6.2055
HybS+mSOS       8    670    2265    160     0.934789   2.2965    0.0047     0.934744  5.48e-01  1.00e+00  3.47e+02  5.22e+02   0.0043  16.1067

HybS+Saw        4    390    1525     80     0.951869   0.5113    0.0021     0.951849  5.48e-01  1.00e+00  2.16e+03  4.23e+03   0.0038  13.3609
HybS+Saw        6    530    2105    120     0.956386   0.0391    0.0099     0.956291  5.48e-01  1.00e+00  4.57e+02  7.57e+02   0.0046  33.1035

This branch

Network: 10 nodes, 13 edges, 3 cost segments
Generators: 5, Demands: 5
Loss coefficients (a, b, c) per generator:
  n1: a=0.007287  b=0.005968  c=0.003850
  n2: a=0.001094  b=0.009002  c=0.007028
  n3: a=0.007897  b=0.008552  c=0.003001
  n4: a=0.008179  b=0.002530  c=0.009637
  n5: a=0.009778  b=0.009934  c=0.009099

Solver logs: /home/runner/work/InfrastructureOptimizationModels.jl/InfrastructureOptimizationModels.jl/branch/test/performance/logs/solver_2026-07-24T18-25-35.log

==============================================================================================================================================
Bilinear Approximation Benchmarks
  Refinement = depth for all methods
==============================================================================================================================================
Method          R   Vars Constrs   Bins    Objective   Gap(%) MIPGap(%)     LowerBnd  rmse δbi   max δbi   rmse δq    max δq  build_t  solve_t
----------------------------------------------------------------------------------------------------------------------------------------------
NLP (Ipopt)     -     40     105      0     0.956760        -         -            -  0.00e+00  0.00e+00  0.00e+00  0.00e+00   0.0320   0.0044

NLP (Uno)       -     40     105      0     0.956760   0.0000         -            -  0.00e+00  0.00e+00  0.00e+00  0.00e+00   0.0013   0.0011

Bin2+sSOS       4    190     535      0     1.180700  23.4061    0.0000     1.180700  8.07e-02  1.41e-01  2.49e+01  5.48e+01   0.0051   1.7703
Bin2+sSOS       6    250     655      0     1.098175  14.7806    0.0000     1.098175  1.26e-01  2.86e-01  5.47e+00  9.07e+00   0.0062   7.3565
Bin2+sSOS       8    310     775      0     1.053069  10.0661    0.0088     1.052976  8.62e-02  2.43e-01  4.18e+01  9.32e+01   0.0058  26.1449

Bin2+mSOS       4    310     805    120     1.180700  23.4061    0.0000     1.180700  8.07e-02  1.41e-01  2.49e+01  5.48e+01   0.0025   1.7241
Bin2+mSOS       6    430    1045    180     1.098175  14.7806    0.0092     1.098074  1.26e-01  2.86e-01  5.47e+00  9.07e+00   0.0025   7.3531
Bin2+mSOS       8    550    1285    240     1.053069  10.0661    0.0045     1.053021  8.62e-02  2.43e-01  4.18e+01  9.32e+01   0.0025  17.6782

Bin2+Saw        4    310    1075    120     0.985823   3.0376    0.0000     0.985823  1.08e-01  2.49e-01  1.11e+02  2.44e+02   0.0471   5.8148
Bin2+Saw        6    430    1495    180     0.958677   0.2004    0.0077     0.958603  7.52e-02  1.68e-01  8.94e+01  1.71e+02   0.0036  13.3775
Bin2+Saw        8    550    1915    240     0.956996   0.0247    0.0072     0.956927  7.52e-02  1.68e-01  1.37e+02  2.09e+02   0.0043  49.6631

HybS+sSOS       4    310    1165      0     0.812261  15.1030    0.0000     0.812261  5.52e-01  1.00e+00  2.04e+02  2.97e+02   0.0068   2.6393
HybS+sSOS       6    410    1545      0     0.891208   6.8515    0.0000     0.891208  5.49e-01  1.00e+00  3.69e+02  7.09e+02   0.0073   4.8752
HybS+sSOS       8    510    1925      0     0.934789   2.2965    0.0087     0.934707  5.48e-01  1.00e+00  3.47e+02  5.22e+02   0.0078  15.4868

HybS+mSOS       4    390    1345     80     0.812261  15.1030    0.0046     0.812224  5.52e-01  1.00e+00  2.04e+02  2.97e+02   0.0033   2.8764
HybS+mSOS       6    530    1805    120     0.891208   6.8515    0.0000     0.891208  5.49e-01  1.00e+00  3.69e+02  7.09e+02   0.0039   6.1807
HybS+mSOS       8    670    2265    160     0.934789   2.2965    0.0047     0.934744  5.48e-01  1.00e+00  3.47e+02  5.22e+02   0.0045  16.0090

HybS+Saw        4    390    1525     80     0.951869   0.5113    0.0021     0.951849  5.48e-01  1.00e+00  2.16e+03  4.23e+03   0.0041  13.4279
HybS+Saw        6    530    2105    120     0.956386   0.0391    0.0099     0.956291  5.48e-01  1.00e+00  4.57e+02  7.57e+02   0.0047  33.0941
HybS+Saw        8    670    2685    160     0.956734   0.0027    0.0014     0.956721  5.48e-01  1.00e+00  6.08e+00  1.07e+01   0.0056 105.8274

DNMDT           4    395    1640     80     0.954878   0.1967    0.0004     0.954874  1.39e-03  3.13e-03  4.70e-04  1.05e-03   0.0031   3.8364
DNMDT           6    550    2315    120     0.956636   0.0130    0.0098     0.956541  6.75e-05  1.57e-04  4.26e+03  9.54e+03   0.0035  16.9305
DNMDT           8    705    2990    160     0.956754   0.0007    0.0098     0.956660  4.62e-06  1.14e-05  9.42e+01  1.49e+02   0.0038  48.2195

==============================================================================================================================================

Container Build (Main)

==============================================================================
Container build-time benchmark
==============================================================================
Leaf container specs  (time_steps = 1:24, segments = 1:5)
------------------------------------------------------------------------------
shape                         N_names        ns/call     bytes/call
------------------------------------------------------------------------------
dense VariableRef                  50         7477.7          26392
dense Float64                      50         6783.0          16792
sparse 2-axis                      50          401.3            848
sparse 3-axis                      50          555.6           1104
dense VariableRef                 500        37141.3         216744
dense Float64                     500        31313.7         120744
sparse 2-axis                     500          384.0            848
sparse 3-axis                     500          550.0           1104
dense VariableRef                5000       905874.0        2293144
dense Float64                    5000       614276.0        1333144
sparse 2-axis                    5000          400.6            848
sparse 3-axis                    5000          551.9           1104
------------------------------------------------------------------------------

End-to-end add_*_container!  (time_steps = 1:24, segments = 1:5)
------------------------------------------------------------------------------
wrapper                       N_names        ns/call     bytes/call
------------------------------------------------------------------------------
add_variable (dense)               50         8656.3          26536
add_constraints (sparse)           50         1459.2           1296
add_variable (dense)              500        35918.0         216888
add_constraints (sparse)          500         1427.7           1296
add_variable (dense)             5000       480230.2        2293288
add_constraints (sparse)         5000         1431.3           1296
------------------------------------------------------------------------------

==============================================================================

Container Build (This branch)

==============================================================================
Container build-time benchmark
==============================================================================
Leaf container specs  (time_steps = 1:24, segments = 1:5)
------------------------------------------------------------------------------
shape                         N_names        ns/call     bytes/call
------------------------------------------------------------------------------
dense VariableRef                  50         8316.5          26392
dense Float64                      50         6798.7          16792
sparse 2-axis                      50          386.5            848
sparse 3-axis                      50          558.8           1104
dense VariableRef                 500        36472.5         216744
dense Float64                     500        29442.9         120744
sparse 2-axis                     500          393.1            848
sparse 3-axis                     500          555.4           1104
dense VariableRef                5000       902980.7        2293144
dense Float64                    5000       583302.7        1333144
sparse 2-axis                    5000          418.4            848
sparse 3-axis                    5000          561.7           1104
------------------------------------------------------------------------------

End-to-end add_*_container!  (time_steps = 1:24, segments = 1:5)
------------------------------------------------------------------------------
wrapper                       N_names        ns/call     bytes/call
------------------------------------------------------------------------------
add_variable (dense)               50         8617.1          26536
add_constraints (sparse)           50         1360.9           1296
add_variable (dense)              500        34711.6         216888
add_constraints (sparse)          500         1380.9           1296
add_variable (dense)             5000       643632.2        2293288
add_constraints (sparse)         5000         1417.4           1296
------------------------------------------------------------------------------

==============================================================================

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants