Weight-Decay
Regularization, Neural Network Training Dynamics
Stars
14
Ecosystems:
Python
Three Mechanisms of Weight Decay Regularization
This repo contains the official implementations of Three Mechanisms of Weight Decay Regularization.
- The config file for the experiments are under the directory of
configs/. - The modified optimization algorithms are in
libs/.
Citation
To cite this work, please use
@inproceedings{
zhang2018three,
title={Three Mechanisms of Weight Decay Regularization},
author={Guodong Zhang and Chaoqi Wang and Bowen Xu and Roger Grosse},
booktitle={International Conference on Learning Representations},
year={2019},
url={https://openreview.net/forum?id=B1lz-3Rct7},
}
Requirements
This project uses Python 3.5.2. Before running the code, you have to install
How to run?
# example
$ python main.py --config configs/cifar100/resnet32/kfac/mb128_lr5e2_damp1e3_cov95_bn_aug.json
Credit
This repo uses a modified version of Tensorflow K-FAC.
Contact
If you have any questions or suggestions about the code or paper, please do not hesitate to contact with Guodong Zhang([email protected] or [email protected]) and Chaoqi Wang([email protected] or [email protected]).