ViViT is a collection of numerical tricks to efficiently access curvature from the generalized Gauss-Newton (GGN) matrix based on its low-rank structure.
pip install vivit-for-pytorch@git+https://github.com/f-dangel/vivit.git#egg=vivit-for-pytorch
It is designed to be used with BackPACK and can compute
GGN eigenvalues (basic example)
GGN eigenpairs (eigenvalues + eigenvector, basic example)
1ˢᵗ- and 2ⁿᵈ-order directional derivatives along GGN eigenvectors (basic example)
Directionally damped Newton steps (basic example)
These operations can further approximate the GGN to reduce cost via sub-sampling, Monte-Carlo approximation, and block-diagonal approximation.