CommonMath Polished Docs

[lukelowry]

Description

Closes #513 , @shakedregev, please let me know if this resolves your concerns. I like it much better now too.

@pelesh Please read the Ramp section; I provide a better explanation of what I was trying to say earlier.

Proposed changes

Better description soon

Checklist

N/A

Further comments

Do we want the .tex files I used to generate these tracked in the repo?

[pelesh]

I would use the same organization as before, just distinguish between discrete function and smooth approximation.

For example, in the section "Step Function" I would define Heaviside function as used in most commercial tools today and after that definition I would explain we are using sigmoid function to approximate the step function. Minimal changes to your original documentation.

Describing all nonsmooth functions in one section and their smooth counterparts in the other section makes documentation harder to read.

[lukelowry]

Corrected, I think it is good to go

[nkoukpaizan]

Please see my suggestion for the sigmoid and update the ramp and quadratic ramp functions accordingly. The issue raised in #513 is about using precise mathematical terminology. There are discrete functions and smooth approximations to those function.

I wouldn't use ReLU and ReQU in this context, as they are terminology used for activation functions.

[lukelowry]

Thank you for suggestions @nkoukpaizan. All applied

[lukelowry]

Oh also @nkoukpaizan the models that consume these functions do use these as activation functions primarily, particularly when we use it for rate limiters. Not an important distinction but worth clarifying

[nkoukpaizan]

Oh also @nkoukpaizan the models that consume these functions do use these as activation functions primarily, particularly when we use it for rate limiters. Not an important distinction but worth clarifying

In that case, please add to the documentation. It is currently not clear to me.

[lukelowry]

Okay, resolved. I just made a more accurate introductory paragraph describing the general application of these functions thusfar.

[nkoukpaizan]

A couple more suggestions. Will wait for @shakedregev to review.

[lukelowry]

Thank you @pelesh @shakedregev @nkoukpaizan for the attention to detail. Implementing these models is basically infeasible without this doc, it is like my Rosetta Stone between the classical representation of these models and our framework