Part VI: Handling Parameter Constraints of Exponential Family In Natural-gradient Methods
Warning: working in Progress (Part VI is incomplete)
Blog posts
2021
Part V: Efficient Natural-gradient Methods for Exponential Family
Warning: working in Progress (incomplete)
Part IV: Natural Gradient Descent and its Extension—Riemannian Gradient Descent
Warning: working in Progress (incomplete)
Part III: Invariance of Natural-Gradients
Goal (edited: 01-Jan-23) This blog post focuses on invariant properties of natural-gradients. We will discuss transformation rules of natural-gradients, ...
Part II: Derivation of Natural-gradients
Goal (edited: 01-Jan-23) This blog post focuses on the derivation of natural-gradients, which are known as Riemannian gradients with the Fisher-Rao metric. W...
Part I: Smooth Manifolds with the Fisher-Rao Metric
Goal (edited: 01-Jan-23) This blog post focuses on the Fisher-Rao metric, which gives rise to the Fisher information matrix (FIM). We will introduce the foll...
Structured Natural Gradient Descent (ICML 2021)
More about this work [1]: Long talk, (Youtube) short talk, extended paper, short paper, poster