Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 24 additions & 3 deletions src/basicfuns.jl
Original file line number Diff line number Diff line change
Expand Up @@ -262,7 +262,7 @@ This is also called the ["softplus"](https://en.wikipedia.org/wiki/Rectifier_(ne
transformation, being a smooth approximation to `max(0,x)`. Its inverse is [`logexpm1`](@ref).

This is also called the ["softplus"](https://en.wikipedia.org/wiki/Rectifier_(neural_networks))
transformation (in its default parametrization, see [`softplus`](@ref)), being a smooth approximation to `max(0,x)`.
transformation (in its default parametrization, see [`softplus`](@ref)), being a smooth approximation to `max(0,x)`.

See:
* Martin Maechler (2012) [“Accurately Computing log(1 − exp(− |a|))”](http://cran.r-project.org/web/packages/Rmpfr/vignettes/log1mexp-note.pdf)
Expand Down Expand Up @@ -364,6 +364,27 @@ Return `log(2 - exp(x))` evaluated carefully.
"""
log2mexp(x::Real) = log1p(-expm1(x))

# Split log(2) into hi + lo, where hi = T(log 2) and lo is the rounding residual in T.
Base.@assume_effects :foldable function _log2_hilo(::Type{T}) where {T<:AbstractFloat}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:foldable asserts :consistent, :effect_free and :terminates, but this allocates BigFloats with finalizers and calls MPFR under setprecision. If const-eval doesn't happen, this runs on every call. If the assumption is wrong, it's UB. Hard-coded constants (like _log1pexp_thresholds_*) avoid all of this. Also: log2 in the name suggests the base-2 logarithm; maybe _logtwo_*? And 2p + a few guard bits is enough, rather than 4p + 64.

setprecision(BigFloat, 4 * precision(T) + 64) do
l2 = log(BigFloat(2))
hi = T(l2)
lo = T(l2 - BigFloat(hi))
return hi, lo
end
end

function log2mexp(x::T) where {T<:Union{Float16,Float32,Float64}}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Restricting this to Union{Float16,Float32,Float64} leaves BigFloat & co. (and the float(x) funnel for Int/Rational) on the inaccurate path. The algorithm itself is type-agnostic; only the lo constant is type-specific.

if x > T(0.5)
log2_hi, log2_lo = _log2_hilo(T)
d = (log2_hi - x) + log2_lo # log(2) - x, accurate to ~eps relative
d < 0 && return T(NaN) # x > log(2): outside the domain

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This changes the behaviour for x > log 2 from DomainError to NaN, only for these three types. I'd drop the check: log(-2 * expm1(-d)) already throws a DomainError for d < 0.

return log(-2 * expm1(-d)) # d == 0 gives -Inf
else
return log1p(-expm1(x)) # generic method, fine away from log(2)
end
end

"""
$(SIGNATURES)

Expand Down Expand Up @@ -391,8 +412,8 @@ _logexpm1(x::BigFloat) = x > 1 ? x + log1p(-exp(-x)) : log(expm1(x))
"""
$(SIGNATURES)

The generalized `softplus` function (Wiemann et al., 2024) takes an additional optional parameter `a` that control
the approximation error with respect to the linear spline. It defaults to `a=1.0`, in which case the softplus is
The generalized `softplus` function (Wiemann et al., 2024) takes an additional optional parameter `a` that control
the approximation error with respect to the linear spline. It defaults to `a=1.0`, in which case the softplus is
equivalent to [`log1pexp`](@ref).

See:
Expand Down
5 changes: 5 additions & 0 deletions test/basicfuns.jl
Original file line number Diff line number Diff line change
Expand Up @@ -205,6 +205,11 @@ end
@test @inferred(log2mexp(T(0))) isa T
@test iszero(log2mexp(T(0)))
@test log2mexp(-T(1)) ≈ log(2 - exp(-T(1)))
for m in 1:5
x = log(T(2))
x -= ldexp(eps(x), m)
@test log2mexp(x) ≈ T(log2mexp(BigFloat(x)))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • ≈ uses rtol = √eps (0.03 for Float16), which is weak.
  • The reference reuses the code under test; log(2 - exp(big(x))) would be independent.
  • Coverage is missing for m = 0 (T(log(2)) lies above log 2 for Float16/Float32 and below for Float64), x within 1 ulp, the DomainError for x > log 2, and BigFloat.
  • Could use the existing ulp_error/ulp_error_maximum helpers, e.g. exhaustively for Float16.
  • The BigFloat accuracy testset scales tolerances by the condition number, which is exactly why it never caught log2mexp(x) accuracy vanishes around the right edge of the domain, at log(2) #102. Maybe add an unscaled check near log 2.

end
end
end

Expand Down
Loading