optimize: Change Hessian function signature?
Created by: btracey
I've been thinking for a while that we might want to change the signature for the Hessian method. The current signature forces the Hessian to be a *SymDense
, while for many problems the Hessian is sparse, or block structured, or something. Directly, this limits the size of problems that can be solved due to memory constraints. Indirectly, this also significantly limits optimizations available. For instance, right now Newton
calls Solve, which means the matrix is always using a Dense
routine. There could be simplifications available. For instance, Newton
really only requires a SolveGradToer
, something like
type SolveVecToer interface {
Symmetric()
SolveGradTo(v []float64) err
}
which is easily implemented by SymDense
, but also enables taking advantage of sparse structure (or approximate solves).
cc @vladimir-ch