--------------------------------------------------------------------------------
Description
--------------------------------------------------------------------------------
Automatic differentiation class for forward mode automatic differentiation using
operator overloading and reimplemented math functions. Single and partial
derivatives are supported.
--------------------------------------------------------------------------------
Available operators
--------------------------------------------------------------------------------
The following operators can be used with Adiff objects:
Operator Description
-------- ----------------------------------------------------------
+ Adiff1 + Adiff2, Adiff + float, float + Adiff
- Adiff1 - Adiff2, Adiff - float, float - Adiff, -Adiff
* Adiff1 * Adiff2, Adiff * float, float * Adiff
/ Adiff1 / Adiff2, Adiff / float, float / Adiff
** Adiff1 ** Adiff2, Adiff ** float, float ** Adiff
+= Adiff1 += Adiff2, Adiff1 += float
-= Adiff1 -= Adiff2, Adiff1 -= float
*= Adiff1 *= Adiff2, Adiff1 *= float
/= Adiff1 /= Adiff2, Adiff1 /= float
**= Adiff1 **= Adiff2, Adiff1 **= float
--------------------------------------------------------------------------------
Available functions
--------------------------------------------------------------------------------
The following functions can be used with Adiff objects:
Function Description
-------- --------------------------
sin sine
cos cosine
tan tangent
asin inverse sine
acos inverse cosine
atan inverse tangent
sqrt square root
exp exponential (e^x)
log natural logarithm
log10 base 10 logarithm
sinh hyperbolic sine
cosh hyperbolic cosine
tanh hyperbolic tangent
asinh inverse hyperbolic sine
acosh inverse hyperbolic cosine
atanh inverse hyperbolic tangent
These functions are not members of the Adiff class, but instead are imported
from the autodiff module. The standard versions of these functions in the math
module will not work with Adiff objects. Be careful not to introduce namespace
collisions by importing any of these functions from both modules. You must
rename one or both of the imported functions that have the same name to avoid
this problem.
--------------------------------------------------------------------------------
Examples
--------------------------------------------------------------------------------
The following is an example of inputs for computing the derivative of a function
of one variable. Comments are included to explain the inputs.
>>> # Import needed class and function from autodiff module
>>> from autodiff import Adiff, cosh
>>>
>>> # Define x as an Adiff object. It has a value of 0.3 and a derivative
>>> # dx/dx = 1.
>>> x = Adiff(val=0.3, derv=1.)
>>>
>>> # Write y as a function of x. The value and derivative with respect to x
>>> # are automatically computed.
>>> y = cosh(-x)
>>>
>>> # Evaluate y at x = 0.3
>>> y.value()
1.0453385141288605
>>>
>>> # Evaluate dy/dx at x = 0.3
>>> y.derivative()
0.3045202934471426
The important thing in this example is the initialization of x with derv=1. If
this step was left out, the derivative of x would be set to 0, which means that
x is a constant. Then the derivative of y would also be 0. When writing
functions with automatic differentiation, always set the derivative of your
independent variable equal to 1. This is equivalent to saying dx/dx = 1, which
is, of course, what you want. The derivative can be set in the constructor, as
in the above example, or using the setDerivative() method of the Adiff class.
Derivatives and values are propagated through assignments. As an example, let's
get the derivative of a long function with respect to x by breaking it up into
a few smaller functions. The long function is:
f(x) = sinh(tan(-x)**(3.*x)) / log(sqrt(exp(x)))
This could, of course, be written in a single line in the code, but usually
practical functions encountered in computer codes are written over several
lines. f(x) and its derivative are evaluated below at x = -0.5 using this
approach:
>>> # Import needed class and functions from autodiff module
>>> from autodiff import Adiff, sinh, tan, log, sqrt, exp
>>>
>>> # Define x as an Adiff object. It has a value of -0.5 and a derivative
>>> # dx/dx = 1.
>>> x = Adiff(val=-0.5, derv=1.)
>>>
>>> # An intermediate function g(x)
>>> g = tan(-x)
>>>
>>> # Another intermediate function h(g(x))
>>> h = g**(3.*x)
>>>
>>> # Another intermediate function p(h(g(x)))
>>> p = sinh(h)
>>>
>>> # Another intermediate function q(x)
>>> q = exp(x)
>>>
>>> # Another intermediate function r(q(x))
>>> r = sqrt(q)
>>>
>>> # Another intermediate function s(r(q(x)))
>>> s = log(r)
>>>
>>> # Finally, f(x)
>>> f = p/s
>>> f.value()
-23.632638563231385
>>> f.derivative()
-151.2311050949442
Adiff supports partial derivatives as well as single derivatives. The usage is
very similar for partial derivatives, except that each Adiff object has
multiple derivatives stored, which are referenced using an index. You can use
as many partial derivatives as you want. Below is a simple example with three
partial derivatives.
>>> # Import needed class and function from autodiff module
>>> from autodiff import Adiff, sin, cos
>>>
>>> # Define x, its value, and the number of derivatives
>>> x = Adiff(val=5.0, nderv=3)
>>>
>>> # Partial of x with respect to x
>>> x.setDerivative(1.0, idx=0)
>>>
>>> # Partial of x with respect to y and z
>>> x.setDerivative(0.0, idx=1)
>>> x.setDerivative(0.0, idx=2)
>>>
>>> # Derivatives can also be initialized with a list. Here, dy/dx = 0,
>>> # dy/dy = 1, and dy/dz = 0. y will be evaluated at -2.
>>> y = Adiff(val=-2.)
>>> y.setAllDerivatives([0.0, 1.0, 0.0])
>>>
>>> # You don't need to set values in the constructor if you don't want to.
>>> # Here, z is set to 0.1 and dz/dz = 1 while all other partials are = 0.
>>> z = Adiff()
>>> z.setValue(0.1)
>>> z.setNumDerivatives(3)
>>> z.setDerivative(0.0, idx=0)
>>> z.setDerivative(0.0, idx=1)
>>> z.setDerivative(1.0, idx=2)
>>>
>>> # Let's evaluate the gradient of f(x,y,z) = sin(x/y) + cos(z)
>>> f = sin(x/y) + cos(z)
>>> f.value()
0.39653202117406927
>>>
>>> # df/dx, df/dy, df/dz, respectively
>>> f.derivative(0)
0.40057180777346685
>>> f.derivative(1)
1.001429519433667
>>> f.derivative(2)
-0.09983341664682815
