From d18bf3eb01a0227c06389bbd226725ca74358d42 Mon Sep 17 00:00:00 2001 From: Luke Lowery Date: Mon, 3 Aug 2026 15:45:09 -0500 Subject: [PATCH 01/10] CommonMath polished docs --- GridKit/CommonMath.md | 161 ++++++++++++------ .../Model/EMT/Component/Load/LoadZ/README.md | 5 +- 2 files changed, 111 insertions(+), 55 deletions(-) diff --git a/GridKit/CommonMath.md b/GridKit/CommonMath.md index 74e79289e..8eff620c2 100644 --- a/GridKit/CommonMath.md +++ b/GridKit/CommonMath.md @@ -2,76 +2,119 @@ Smooth, autodiff-friendly replacements for piecewise functions used across GridKit component models. See `CommonMath.hpp` for implementation details. -## Primitives - -The scale $\mu=4\cdot f_{\text{sync}}=240$ is chosen so $\sigma$ behaves like a step on inputs while keeping derivatives finite. As $\mu \to \infty$, these functions approach their exact targets. - -| Name | Description | Usage | -|------|-------------|-------| -| `sigmoid` | Step function | `GENSAL`, `GENROU`, `REECA` | -| `ramp` | Smooth one-sided ramp | `REGCA`, `REECA`, `REPCA` | -| `qramp` | Exact one-sided quadratic ramp | `IEEET1` | +- [Primitives](#primitives) +- [Smooth Primitives](#smooth-primitives) +- [Derived Functions](#derived-functions) -### $\sigma$ - `sigmoid` +## Primitives -The sigmoid is also known as the logistic function. The equivalent `tanh` form is used for numerical stability because the exponential form can divide by a very large value. +| Symbol | Name | Description | +|--------|------|-------------| +| $H$ | Heaviside | Unit step | +| $\text{ReLU}$ | Rectified linear unit | One-sided ramp | +| $\text{ReQU}$ | Rectified quadratic unit | One-sided quadratic ramp | ```math \begin{aligned} - \sigma(x) + H(x) &= \begin{cases} 0 & x\le 0 \\[0pt] 1 & x\gt 0 \end{cases} \\[0pt] - &\approx \dfrac{1}{2}\left(1+\tanh\left(\dfrac{\mu x}{2}\right)\right) + \text{ReLU}(x) + &= + \begin{cases} + 0 & x\le 0 \\[0pt] + x & x\gt 0 + \end{cases} \\[0pt] + \text{ReQU}(x) + &= + \begin{cases} + 0 & x\le 0 \\[0pt] + x^2 & x\gt 0 + \end{cases} \\[0pt] +\end{aligned} +``` + +## Smooth Primitives + +| Symbol | Name | API | Description | +|--------|------|-----|-------------| +| $\sigma$ | Logistic Function | `sigmoid` | Smooth Heaviside | +| $\rho$ | Ramp | `ramp` | Smooth ReLU | +| $q$ | Quadratic Ramp | `qramp` | Smooth ReQU | + + +The default scale $\mu$ of these functions is chosen such that $\sigma'(0)=f_{\text{sync}}$, where $f_{\text{sync}}$ is the fundamental frequency of the `PhasorDynamics` model. As $\mu\to\infty$, $\rho$ and $q$ converge to $\text{ReLU}$ and $\text{ReQU}$ everywhere, while $\sigma$ converges to $H$ for $x\ne0$. + +### Logistic Function + +The logistic function is evaluated using the equivalent hyperbolic-tangent form below to avoid overflow when evaluating the exponential form directly. + +```math +\begin{aligned} + H(x) &\approx \sigma(x) = \dfrac{1}{2}\left(1+\tanh\left(\dfrac{\mu x}{2}\right)\right) \end{aligned} ``` ![](../docs/Figures/CommonMath/sigmoid.svg) -### $\rho$ - `ramp` +### Ramp -`ramp` is the softplus approximation to the one-sided ramp. We do not use $x\sigma(x)$ directly because it introduces a negative tail for $x \lt 0$, while softplus stays nonnegative and approaches $\max(x, 0)$ as the smoothing becomes sharp. +Also known as $\text{softplus}(x)$, the Ramp function is a non-negative approximation of $\text{ReLU}(x)$. We do not use $x\,\sigma(x)$ because it introduces a negative tail for $x \lt 0$, while $\text{softplus}(x)$ stays nonnegative. ```math \begin{aligned} - \rho(x) - &= x\,\sigma(x) \\[0pt] - &\approx \dfrac{x+\lvert x\rvert}{2}+\dfrac{\ln\!\left(1+e^{-\mu\lvert x\rvert}\right)}{\mu} + \text{ReLU}(x) + &\approx \rho(x) + = \dfrac{1}{\mu}\ln(1+e^{\mu x}) \end{aligned} ``` ![](../docs/Figures/CommonMath/ramp.svg) -### $q$ - `qramp` +Although $\rho(x)$ is real-analytic, the implemented form is an overflow-safe representation of the function: -*Note*: the implementation of the quadratic ramp `q(x)` could be optimized with Enzyme features down the road so that we don't need the smooth approximation. +```math +\begin{aligned} + \rho(x) = + \dfrac{x+\lvert x\rvert}{2} + + \dfrac{1}{\mu}\ln\left(1+e^{-\mu\lvert x\rvert}\right) +\end{aligned} +``` +The kinks of the two terms cancel exactly. + +### Quadratic Ramp + +This approximation is implemented using the logistic function. ```math -q(x)=x^2\,\sigma(x) +\begin{aligned} + \text{ReQU}(x) &\approx q(x) = x^2\,\sigma(x) +\end{aligned} ``` ![](../docs/Figures/CommonMath/qramp.svg) ## Derived Functions -| Name | Description | Usage | -|------|-------------|-------| -| `max` | Smooth binary maximum | `REGCA`, `REECA`, `REECB` | -| `min` | Smooth binary minimum | `REGCA`, `REECA` | -| `clamp` | Bounded saturation | `IEEEST`, `REGCA`, `REECA`, `REECB`, `REPCA` | -| `deadband1` | Type 1 no-offset signed two-sided deadband | - | -| `deadband2` | Type 2 offset signed two-sided deadband | `REECA`, `REECB`, `REPCA` | -| `slew` | Symmetric slew-rate limiter | - | -| `linseg` | Saturated linear segment contribution | `REGCA`, `REECA` | -| `above` | Above-lower-limit indicator | `REPCA` | -| `below` | Below-upper-limit indicator | - | -| `inside` | Interior pulse indicator | - | -| `outside` | Outside-band indicator | `REECA`, `REECB` | -| `antiwindup` | Anti-windup limited derivative | `IEEET1`, `SEXS-PTI`, `TGOV1`, `REECA`, `REECB`, `REPCA` | - -### `max` +| Name | API | Description | +|------|-----|-------------| +| Maximum | `max` | Smooth binary maximum | +| Minimum | `min` | Smooth binary minimum | +| Clamp | `clamp` | Bounded saturation | +| Type 1 Deadband | `deadband1` | No-offset signed two-sided deadband | +| Type 2 Deadband | `deadband2` | Offset signed two-sided deadband | +| Slew | `slew` | Symmetric slew-rate limiter | +| Linear Segment | `linseg` | Saturated linear segment contribution | +| Above | `above` | Above-lower-limit indicator | +| Below | `below` | Below-upper-limit indicator | +| Inside | `inside` | Interior pulse indicator | +| Outside | `outside` | Outside-band indicator | +| Antiwindup | `antiwindup` | Anti-windup limited derivative | + +### Maximum ```math \begin{aligned} @@ -87,7 +130,7 @@ q(x)=x^2\,\sigma(x) ![](../docs/Figures/CommonMath/max.svg) -### `min` +### Minimum ```math \begin{aligned} @@ -103,7 +146,9 @@ q(x)=x^2\,\sigma(x) ![](../docs/Figures/CommonMath/min.svg) -### `clamp` +### Clamp + +The limits satisfy $\ell\le u$. ```math \begin{aligned} @@ -120,7 +165,9 @@ q(x)=x^2\,\sigma(x) ![](../docs/Figures/CommonMath/clamp.svg) -### `deadband1` +### Type 1 Deadband + +The limits satisfy $\ell\le u$. ```math \begin{aligned} @@ -137,7 +184,9 @@ q(x)=x^2\,\sigma(x) ![](../docs/Figures/CommonMath/deadband1.svg) -### `deadband2` +### Type 2 Deadband + +The limits satisfy $\ell\le u$. ```math \begin{aligned} @@ -154,7 +203,9 @@ q(x)=x^2\,\sigma(x) ![](../docs/Figures/CommonMath/deadband2.svg) -### `slew` +### Slew + +The rate limit satisfies $r\ge0$. ```math \begin{aligned} @@ -171,7 +222,9 @@ q(x)=x^2\,\sigma(x) ![](../docs/Figures/CommonMath/slew.svg) -### `linseg` +### Linear Segment + +The breakpoints satisfy $a +The limits satisfy $\ell\le u$. ```math \begin{aligned} diff --git a/GridKit/Model/EMT/Component/Load/LoadZ/README.md b/GridKit/Model/EMT/Component/Load/LoadZ/README.md index 14d46f0c6..15c9119af 100644 --- a/GridKit/Model/EMT/Component/Load/LoadZ/README.md +++ b/GridKit/Model/EMT/Component/Load/LoadZ/README.md @@ -33,9 +33,8 @@ $\mathbf{z}$ | Impedance | [VectorFit](../../../Operators/Rational/VectorFit/REA ### Submodel Validation -The current is differential for a nonsingular linear coefficient and algebraic -when the coefficient is zero. Partially singular coefficients are not -supported. +The current is differential when the linear coefficient has full rank and +algebraic when it is the zero matrix. No other coefficients are supported. ```math \mathbf{E}^{\mathbf{z}}=\mathbf{0} From eeb0cbdb9e5384b0685e83aa92e4a549d4a72768 Mon Sep 17 00:00:00 2001 From: lukelowry Date: Wed, 5 Aug 2026 21:07:35 -0500 Subject: [PATCH 02/10] Fix CommonMath trailing whitespace --- GridKit/CommonMath.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/GridKit/CommonMath.md b/GridKit/CommonMath.md index 8eff620c2..f2d464ebc 100644 --- a/GridKit/CommonMath.md +++ b/GridKit/CommonMath.md @@ -67,7 +67,7 @@ Also known as $\text{softplus}(x)$, the Ramp function is a non-negative approxim ```math \begin{aligned} \text{ReLU}(x) - &\approx \rho(x) + &\approx \rho(x) = \dfrac{1}{\mu}\ln(1+e^{\mu x}) \end{aligned} ``` @@ -78,7 +78,7 @@ Although $\rho(x)$ is real-analytic, the implemented form is an overflow-safe re ```math \begin{aligned} - \rho(x) = + \rho(x) = \dfrac{x+\lvert x\rvert}{2} + \dfrac{1}{\mu}\ln\left(1+e^{-\mu\lvert x\rvert}\right) \end{aligned} From fd952d0271e7db60225d27c7eb35a6c79f150895 Mon Sep 17 00:00:00 2001 From: lukelowry Date: Thu, 6 Aug 2026 11:09:59 -0500 Subject: [PATCH 03/10] Minor reorganizing and rewording --- GridKit/CommonMath.md | 71 ++++++++++--------- .../Model/EMT/Component/Load/LoadZ/README.md | 4 +- 2 files changed, 41 insertions(+), 34 deletions(-) diff --git a/GridKit/CommonMath.md b/GridKit/CommonMath.md index f2d464ebc..48082e294 100644 --- a/GridKit/CommonMath.md +++ b/GridKit/CommonMath.md @@ -3,42 +3,10 @@ Smooth, autodiff-friendly replacements for piecewise functions used across GridKit component models. See `CommonMath.hpp` for implementation details. - [Primitives](#primitives) -- [Smooth Primitives](#smooth-primitives) - [Derived Functions](#derived-functions) ## Primitives -| Symbol | Name | Description | -|--------|------|-------------| -| $H$ | Heaviside | Unit step | -| $\text{ReLU}$ | Rectified linear unit | One-sided ramp | -| $\text{ReQU}$ | Rectified quadratic unit | One-sided quadratic ramp | - -```math -\begin{aligned} - H(x) - &= - \begin{cases} - 0 & x\le 0 \\[0pt] - 1 & x\gt 0 - \end{cases} \\[0pt] - \text{ReLU}(x) - &= - \begin{cases} - 0 & x\le 0 \\[0pt] - x & x\gt 0 - \end{cases} \\[0pt] - \text{ReQU}(x) - &= - \begin{cases} - 0 & x\le 0 \\[0pt] - x^2 & x\gt 0 - \end{cases} \\[0pt] -\end{aligned} -``` - -## Smooth Primitives - | Symbol | Name | API | Description | |--------|------|-----|-------------| | $\sigma$ | Logistic Function | `sigmoid` | Smooth Heaviside | @@ -50,6 +18,19 @@ The default scale $\mu$ of these functions is chosen such that $\sigma'(0)=f_{\t ### Logistic Function +The Heaviside function $H$ is a unit step, defined: + +```math +\begin{aligned} + H(x) + &= + \begin{cases} + 0 & x\le 0 \\[0pt] + 1 & x\gt 0 + \end{cases} +\end{aligned} +``` + The logistic function is evaluated using the equivalent hyperbolic-tangent form below to avoid overflow when evaluating the exponential form directly. ```math @@ -62,6 +43,19 @@ The logistic function is evaluated using the equivalent hyperbolic-tangent form ### Ramp +The rectified linear unit $\text{ReLU}$ is a one-sided ramp: + +```math +\begin{aligned} + \text{ReLU}(x) + &= + \begin{cases} + 0 & x\le 0 \\[0pt] + x & x\gt 0 + \end{cases} +\end{aligned} +``` + Also known as $\text{softplus}(x)$, the Ramp function is a non-negative approximation of $\text{ReLU}(x)$. We do not use $x\,\sigma(x)$ because it introduces a negative tail for $x \lt 0$, while $\text{softplus}(x)$ stays nonnegative. ```math @@ -87,6 +81,19 @@ The kinks of the two terms cancel exactly. ### Quadratic Ramp +The rectified quadratic unit $\text{ReQU}$ is a one-sided quadratic ramp: + +```math +\begin{aligned} + \text{ReQU}(x) + &= + \begin{cases} + 0 & x\le 0 \\[0pt] + x^2 & x\gt 0 + \end{cases} +\end{aligned} +``` + This approximation is implemented using the logistic function. ```math diff --git a/GridKit/Model/EMT/Component/Load/LoadZ/README.md b/GridKit/Model/EMT/Component/Load/LoadZ/README.md index 15c9119af..566a4c07f 100644 --- a/GridKit/Model/EMT/Component/Load/LoadZ/README.md +++ b/GridKit/Model/EMT/Component/Load/LoadZ/README.md @@ -33,8 +33,8 @@ $\mathbf{z}$ | Impedance | [VectorFit](../../../Operators/Rational/VectorFit/REA ### Submodel Validation -The current is differential when the linear coefficient has full rank and -algebraic when it is the zero matrix. No other coefficients are supported. +The current is differential when $\mathbf{E}^{\mathbf{z}}$ is nonsingular and +algebraic when $\mathbf{E}^{\mathbf{z}}=\mathbf{0}$. ```math \mathbf{E}^{\mathbf{z}}=\mathbf{0} From d0fca4513934ae0f64207ea4d9ca1a0a922c8891 Mon Sep 17 00:00:00 2001 From: Luke Lowery Date: Thu, 6 Aug 2026 13:06:01 -0500 Subject: [PATCH 04/10] Update GridKit/CommonMath.md Co-authored-by: Nicholson Koukpaizan <72402802+nkoukpaizan@users.noreply.github.com> --- GridKit/CommonMath.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/GridKit/CommonMath.md b/GridKit/CommonMath.md index 48082e294..d70c76f33 100644 --- a/GridKit/CommonMath.md +++ b/GridKit/CommonMath.md @@ -9,9 +9,9 @@ Smooth, autodiff-friendly replacements for piecewise functions used across GridK | Symbol | Name | API | Description | |--------|------|-----|-------------| -| $\sigma$ | Logistic Function | `sigmoid` | Smooth Heaviside | -| $\rho$ | Ramp | `ramp` | Smooth ReLU | -| $q$ | Quadratic Ramp | `qramp` | Smooth ReQU | +| $\sigma$ | Logistic Function | `sigmoid` | Smooth approximation to the Heaviside step function | +| $\rho$ | Ramp | `ramp` | Smooth approximation to the ramp function | +| $q$ | Quadratic Ramp | `qramp` | Smooth approximation to the quadratic ramp function | The default scale $\mu$ of these functions is chosen such that $\sigma'(0)=f_{\text{sync}}$, where $f_{\text{sync}}$ is the fundamental frequency of the `PhasorDynamics` model. As $\mu\to\infty$, $\rho$ and $q$ converge to $\text{ReLU}$ and $\text{ReQU}$ everywhere, while $\sigma$ converges to $H$ for $x\ne0$. From 77bd848ae9cb5eb7c8ef73950f1c9bd506ea946c Mon Sep 17 00:00:00 2001 From: Luke Lowery Date: Thu, 6 Aug 2026 13:06:47 -0500 Subject: [PATCH 05/10] Update GridKit/CommonMath.md Co-authored-by: Nicholson Koukpaizan <72402802+nkoukpaizan@users.noreply.github.com> --- GridKit/CommonMath.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/GridKit/CommonMath.md b/GridKit/CommonMath.md index d70c76f33..c3ddf1f73 100644 --- a/GridKit/CommonMath.md +++ b/GridKit/CommonMath.md @@ -18,7 +18,7 @@ The default scale $\mu$ of these functions is chosen such that $\sigma'(0)=f_{\t ### Logistic Function -The Heaviside function $H$ is a unit step, defined: +The Heaviside function $H$ is a unit step, defined as: ```math \begin{aligned} From 5f4db812f73bde34f76733d76a6f7d8c4f5dc68b Mon Sep 17 00:00:00 2001 From: Luke Lowery Date: Thu, 6 Aug 2026 13:07:17 -0500 Subject: [PATCH 06/10] Update GridKit/CommonMath.md Co-authored-by: Nicholson Koukpaizan <72402802+nkoukpaizan@users.noreply.github.com> --- GridKit/CommonMath.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/GridKit/CommonMath.md b/GridKit/CommonMath.md index c3ddf1f73..90a45fe69 100644 --- a/GridKit/CommonMath.md +++ b/GridKit/CommonMath.md @@ -31,7 +31,7 @@ The Heaviside function $H$ is a unit step, defined as: \end{aligned} ``` -The logistic function is evaluated using the equivalent hyperbolic-tangent form below to avoid overflow when evaluating the exponential form directly. +We use the logistic function as a smooth approximation to the Heaviside function. This approximation is evaluated using the equivalent hyperbolic-tangent form below to avoid overflow when evaluating the exponential form directly. ```math \begin{aligned} From 1864eb66600404168957827633e0241a8533eeac Mon Sep 17 00:00:00 2001 From: Luke Lowery Date: Thu, 6 Aug 2026 13:07:38 -0500 Subject: [PATCH 07/10] Update GridKit/CommonMath.md Co-authored-by: Nicholson Koukpaizan <72402802+nkoukpaizan@users.noreply.github.com> --- GridKit/CommonMath.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/GridKit/CommonMath.md b/GridKit/CommonMath.md index 90a45fe69..7ba68c101 100644 --- a/GridKit/CommonMath.md +++ b/GridKit/CommonMath.md @@ -35,7 +35,7 @@ We use the logistic function as a smooth approximation to the Heaviside function ```math \begin{aligned} - H(x) &\approx \sigma(x) = \dfrac{1}{2}\left(1+\tanh\left(\dfrac{\mu x}{2}\right)\right) + \sigma(x) = \dfrac{1}{2}\left(1+\tanh\left(\dfrac{\mu x}{2}\right)\right) &\approx H(x) \end{aligned} ``` From 0d0cc6c17b56e9c4fa104d9f0c75dd198aa3b005 Mon Sep 17 00:00:00 2001 From: lukelowry Date: Thu, 6 Aug 2026 15:57:38 -0500 Subject: [PATCH 08/10] clarify activation fucntion application --- GridKit/CommonMath.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/GridKit/CommonMath.md b/GridKit/CommonMath.md index 7ba68c101..32bdd06a8 100644 --- a/GridKit/CommonMath.md +++ b/GridKit/CommonMath.md @@ -1,6 +1,9 @@ # CommonMath -Smooth, autodiff-friendly replacements for piecewise functions used across GridKit component models. See `CommonMath.hpp` for implementation details. +Smooth, autodiff-friendly approximations of piecewise functions used across +GridKit component models. In model equations, these functions act as smooth +activation functions for saturation, limits, deadbands, and antiwindup behavior. See +`CommonMath.hpp` for implementation details. - [Primitives](#primitives) - [Derived Functions](#derived-functions) @@ -60,9 +63,8 @@ Also known as $\text{softplus}(x)$, the Ramp function is a non-negative approxim ```math \begin{aligned} - \text{ReLU}(x) - &\approx \rho(x) - = \dfrac{1}{\mu}\ln(1+e^{\mu x}) + \rho(x) + = \dfrac{1}{\mu}\ln(1+e^{\mu x}) \approx \text{ReLU}(x) \end{aligned} ``` @@ -98,7 +100,7 @@ This approximation is implemented using the logistic function. ```math \begin{aligned} - \text{ReQU}(x) &\approx q(x) = x^2\,\sigma(x) + q(x) = x^2\,\sigma(x) \approx \text{ReQU}(x) \end{aligned} ``` From d55036ae23b025dd2983429e908ceef33d7d2911 Mon Sep 17 00:00:00 2001 From: Luke Lowery Date: Thu, 6 Aug 2026 18:40:24 -0500 Subject: [PATCH 09/10] Update GridKit/CommonMath.md Co-authored-by: Nicholson Koukpaizan <72402802+nkoukpaizan@users.noreply.github.com> --- GridKit/CommonMath.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/GridKit/CommonMath.md b/GridKit/CommonMath.md index 32bdd06a8..cf7e74ba6 100644 --- a/GridKit/CommonMath.md +++ b/GridKit/CommonMath.md @@ -59,7 +59,7 @@ The rectified linear unit $\text{ReLU}$ is a one-sided ramp: \end{aligned} ``` -Also known as $\text{softplus}(x)$, the Ramp function is a non-negative approximation of $\text{ReLU}(x)$. We do not use $x\,\sigma(x)$ because it introduces a negative tail for $x \lt 0$, while $\text{softplus}(x)$ stays nonnegative. +We implement the Ramp function as the $\text{softplus}(x)$, non-negative approximation of $\text{ReLU}(x)$. We do not use $x\,\sigma(x)$ because it introduces a negative tail for $x \lt 0$, while $\text{softplus}(x)$ stays nonnegative. ```math \begin{aligned} From 58c9488b91aa5e2bba10f19d32d972cf65ba01d5 Mon Sep 17 00:00:00 2001 From: Luke Lowery Date: Thu, 6 Aug 2026 18:40:45 -0500 Subject: [PATCH 10/10] Update GridKit/CommonMath.md Co-authored-by: Nicholson Koukpaizan <72402802+nkoukpaizan@users.noreply.github.com> --- GridKit/CommonMath.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/GridKit/CommonMath.md b/GridKit/CommonMath.md index cf7e74ba6..d8f409c85 100644 --- a/GridKit/CommonMath.md +++ b/GridKit/CommonMath.md @@ -96,7 +96,7 @@ The rectified quadratic unit $\text{ReQU}$ is a one-sided quadratic ramp: \end{aligned} ``` -This approximation is implemented using the logistic function. +We implement an approximation to $\text{ReQU}$ using the logistic function. ```math \begin{aligned}