ViViT

ViViT is a collection of numerical tricks to efficiently access curvature from the generalized Gauss-Newton (GGN) matrix based on its low-rank structure.

pip install vivit-for-pytorch@git+https://github.com/f-dangel/vivit.git#egg=vivit-for-pytorch

It is designed to be used with BackPACK and can compute

• GGN eigenvalues (basic example)

• GGN eigenpairs (eigenvalues + eigenvector, basic example)

• 1ˢᵗ- and 2ⁿᵈ-order directional derivatives along GGN eigenvectors (basic example)

• Directionally damped Newton steps (basic example)

These operations can further approximate the GGN to reduce cost via sub-sampling, Monte-Carlo approximation, and block-diagonal approximation.

Getting started

• How to use ViViT
  • Preliminaries
  • Integration with BackPACK

ViViT

• Computations
  • GGN eigenvalues
  • GGN eigenpairs (eigenvalues + eigenvector)
  • 1ˢᵗ- and 2ⁿᵈ-order directional derivatives along GGN eigenvectors
  • Directionally damped Newton steps
• Code samples