Use CppADCodeGen with variable number of independent variable arrays

[fdevinc]

Hi everyone! I am interested in this issue which was raised with regards to CppADCodeGen. I think it would be great if such a feature would be implemented in CppAD!

[bradbell]

I am not sure what you are trying to do here. Would you please provide a simple example.

[fdevinc]

Sure! To the best of my knowledge, CppAD exposes the following interface for its ADFun objects:

ADVector x;        // <---- Array of independent variables.
ADVector y;
...
ADFun adFun(x, y);

where x is an array of independent variables and y is a vector of dependent variables. However, a typical use case features independent variables provided as a vector of independent variable arrays, i.e.:

std::vector<ADVector> xVec;       // <---- Vector of independent variable arrays.
ADVector y;
...
ADFun adFun(xVec, y);

Unfortunately, such interface does not exist for CppAD, nor for CppADCodeGen, and if the independent variables are provided in this way, one must always assemble a single array in order to use the AD functionalities. Is such a change hard to implement? If not, how hard would it be to integrate it with CppADCodeGen?

[fdevinc]

Notice that in CppADCodeGen signatures that support the above interface exist (see this issue), but they seem to not be usable! This gave me a false hope that the feature I am suggesting was available already!

[bradbell]

Will you be taking derivatives with respect to one of the independent variables while the others are held constant, or are you interested in cross term derivatives with respect to all the independent variables ?

[fdevinc]

Possibly both, that is, by computing the Jacobian it would be great to have a tuple containing all the sub-matrices. Similarly, by computing the Hessian it would be great to have a tuple containing all the cross term derivatives. For instance, something as follows:

std::vector<ADVector> xVec{x1, x2};
...
const auto [wrtX1, wrtX2] = model.SparseJacobian(xVec, ...);
const auto [wrtX1X1, wrtX2X2, wrtX1X2] = model.SparseHessian(xVec, ...);

Another great autodiff library employs a similar syntax!

[bradbell]

I suggest creating your own class that has the interface that you want where one or more ADFun objects are data in the class.

One thing I would like to draw your attention to is that in CppAD there are three types of AD values; see the heading at_type on
https://coin-or.github.io/CppAD/doc/atomic_three.htm#ad_type

[fdevinc]

I see! So you suggest to implement an AD function object that contains an ADFun object f_i for each independent variable array where I treat all independent variables x_j with j != i as dynamic parameters? This could work, but it would sacrifice too much efficiency, would it not? Also, using this approach I could not compute cross derivative terms.

[bradbell]

Suppose you are computing the Hessian of f(u,v) w.r.t u at a certain value of (u,v) where u and v are vectors and the value of v changes.
In this case one would want an ADFun object where u is a variable and v is a dynamic parameter.

If you also need the Hessian of f(u,v)( w.r.t v at values of (u,v) perhaps two function objects would be better where the parameter and variable roles switch.

If you also need the Cross partials w.r.t u and v, perhaps it is best to have both u and v variables.

In any case, you could create your own AD function object that implements the interface you want and do the most efficient thing (what every that is) in your object.