deepmd-kit

New features:

• support the plugin for GROMACS (#1160 #1169 )
• support the init-frz-model option for se_r and se_t (#1144)
• support data in hdf5 format (#1163)
• automatic batch size for the inference of DP (#1165 #1154 #1173 )
• Allow to scale LR in parallel training in different ways. (#1167)

Enhancement for developers:

• support descriptor plugin (#1118 #1143 )
• support lammps 20210831 (#1129)

Bug fixings:

• explicitly set lammps' neighbor request to full to fix bug #1109
• Add init-frz-model command support to the frozen model generated by this command (#1137)
• Add error message for repeated model compression (#1136)
• doc related improvements and bug fixings (#1141 #1131 #1147 #1156 #1153 #1171 )
• Fix freezing error on checkpoint from parallel training. (#1166)

New features:

• correct heat flux calculation: interface for deepmd with the centroids atoms, full 3x3 "atomic-virial" (#1093).
• Enable init-frz-model support for the original model (#1102 #1107 )
• support init-frz-model for hybrid descriptor (#1112)

Enhancements:

• use np.testing.assert_almost_equal for array comparing (#1059)
• set allow_growth in default_tf_session_config (#1067)
• Enable parallel training UT in GitHub CI. (#1075)
• create cross-references in docstring (#1083)
• add ABC for descriptors (#1081)
• merge duplicated NeighborStat.get_stat (#1103)

Bug fixings:

• fix hybrid descriptor training error (#1052)
• bugs and memory issues in UTs (#1056 #1066 )
• copy all_virial for float precision (#1069)
• fix building problem on macos (#1071)
• use @loader_path on macos instead of $ORIGIN (#1078)
• Revert "get library extension suffix from built-in method" #1072
• undo reset lcurve.out during the model compression process (#1080)
• fix typo: lcueve.out->lcurve.out (#1077)
• create model compression checkpoint, avoid overwriting original checkpoint (#1076)
• Fix shape mismatch when type_embedding is enabled and type_one_side is disabled (#1074 )
• reduce rcut and sel in the example of se_e3 (#1082)
• Fix a potential slice bug in se_t descriptor (#1087)
• make compress work for hybrid descriptor composed of se_e2_a (#1094)
• Fix gradient not averaged when parallel training. (#1104)
• fix bug of single precision model compression (#1110)
• fix bug of single precision transfer (#1111)
• fix LAMMPS_VERSION_NUMBER condition (#1116)
• Fix missing std::numeric_limits (#1113)
• fix data_modifier OOM problem when set size is too large (#1117)
• fix bugs of dipole charge modifier: binary str and missing frozen node (#1124 )
• fix "Call to method DeepTensor.init with too many arguments" (#1125)

Breaking changes to v1.3

• Training parameters: Several training parameters have been updated. Original training data is splited into training data and validation data. Please read the document to apply the changes. Old styles can still work but are not recommended.
• Model inference: Old models trained by v1 will not work in v2. Run dp convert-from to convert old models to v2.
• Python interface: deepmd.DeepPot has been moved to deepmd.infer.DeepPot.
• C++ interface: NNPInter has been renamed to deepmd::DeepPot and NNPInter.h has been renamed to DeepPot.h. Use -ldeepmd_cc to link instead.

New features

• Model compression (#350 #586 #610 #921 #948 #956 #1000 #1008 #1020 #1043)
• Parallel training (#892 #905 #913 #1030 #1032) (Bytedance)
• ROCm device support (#656 )
• New descriptor: three body embedding (se_e3)
• Hybridization of descriptors (hybrid)
• Type embedding
• Training and inference the dipole (vector) and polarizability (matrix). (#495 #538 #927)
• Support derivatives of the tensor properties. (#805)
• Split of training and validation dataset.
• Model deviation for virial
• Add subcommand and python interface to calculate model-deviation (#715)
• Automatically determine the sel from the training data. (#831)
• Building with lammps with plugin mode (#930 #945)

Performance improvement:

• More efficient training: all customized OPs are implemented with GPU.
• MPI support for atomic model deviation #628
• speedup ROCm kernels which use atomicAdd (#809 #815 ) (from ByteDance)
• speedup CUDA kernels (use atomicAdd inside) by reducing the global memory write (#811)
• speedup tabulate cuda kernel by reducing shm using (#830) (Bytedance)
• speedup format_nlist_b (#832 #845)
• speedup scan_nlist kernel (#1028)

Enhancements

• Strict argument check in the input script.
• Auto conversion of input file to v2.0 compatibility
• Append out_file when lammps restarts #640
• Document and examples for the C++ interface #652 #663
• Instructions for the i-pi #660
• Document for the network size and sel #657
• Use fmod to wrap the coord of atoms (solve slow PBC) (#741)
• bit operations to encode neighbor information
• add CUDA/ROCM buidling documents (#739)
• add type-embedding developer doc (#762 #967)
• add model compression support for models with exclude_types feature (#754)
• improve the doc and user interface of model compression (#772)
• support converting models generated in v1.3 to 2.0 compatibility (#725)
• give a default value to T and convert models from v1.2 to 2.0 compatibility (#789)
• improved documents for conda (#798 #925)
• throw a message if tf runtime is incompatible (#797)
• capture OOM and print debug message (#801)
• add message for DecodeError raised when using model compression (#839)
• Passing error to TF instead of exit (#918)
• refactor docs (#952)
• add an example of nopbc and related docs (#994)
• add dp --version (#995)
• add the argument tensorboard_freq to control sampling ratio during training. (#996)
• add sphinx plugins viewcode and intersphinx (#997)
• generate Python API document automatically (#998)
• give a clear message if model.get_ntypes()<data.get_ntypes() (#1016)
• add docstring for descrpt/se_e2_a (#1017)
• add docstring for fit/ener (#1024)
• add InputNlist into API doc (#1009)
• save checkpoint files with step and keep recent files (#1031)

Improvement of the code for developers

• Support version of the model. Easily check model compatability
• Clear and pythonic python interface
• C++ lib that can be tested independently
• C++ API that can be tested independently
• OP supports multi-device.
• Added deepmd namespace for the C++ API
• UT for Cuda/ROCm code (#569)
• UT for model compression (#586)
• UT for prod_force/virial ops (#703 #741)
• CI test Lammps build (#600)
• allow c++ tests to run without internet (#785)
• build low and high precision at the same time (#879)
• support to specify CUDA/ROCm root in python pkg building (#834) (Bytedance)
• use cached Session to speed up py tests (#833)
• remove cub include for CUDA>=11 (#866)
• Add Errcheck after every kernel function runs And merge redundant code (#855)
• adapt changes to auditwheel directory in manylinux (#889)
• enhance the cli to generate doc json file (#891)
• raise warning before training if sel is not enough (#914)
• make native MD compatible with v2.0 (#950)
• fix type hints and add doc for exclude_types (#1005)
• use TF's built-in method to get numpy dtype (#1035)

Bug fixings:

• Remove using namespace std. Solve compiling compatability problem.
• cuda memory access error #566
• Relative force model deviation is not copied back at single precision #599
• Correct way of allocating memory in float precision #612
• Fix TB logdir remove bug #617
• Illegal nlist #680
• Bug in prod_virial_grad that causes wrong results when training with virials #685
• Uniform random seed #691
• Illegal nlist #680
• Bug in prod_virial_grad that causes wrong results when training with virials #685
• Uniform random seed #691
• fix bug of adding int to a None random seed (#705)
• reuse the zero layer rather than building a new one (#714)
• fix bug in CI (#739)
• fix bug 824 and Synchronize updates to CUDA cod (#828)
• Fix the empty neighbor distance array in neighbor_stat.py (#882)
• fix InvalidArgumentError caused by zero sel and optimize zero matrix (#900)
• fix 'NoneType' has no len() in auto_sel (#911)
• set input DeepmdData.type_map to input type_map (#924)
• Fix member declartion of deepmd and deepmd.entrypoints. (#922)
• add aliases to Arguments (#933)
• fix bug of gelu activation function (#939)
• convert decay_rate to stop_lr from old inputs (#949)
• only enable link what you use on GNU compilers (#962)
• Do not find protobuf for python (#963)
• fix an error in stress by ase interface (#964)
• remove bare except and limit the try clause (#977)
• fix python cmake error (#976)
• Instantiate RunOptions first when training. (#1019)
• Fix complier type in cmake: CMAKE_COMPILER_IS_GNUCXX (#1038)
• other cleanups of the code (#968 #970 #975 #999 #1004 #1002 #1001 #1010 #1014 #1012 #1011 #1021 #1036 #1037)

Contributors

• Han Bao
• Roberto Car
• Junhan Chang
• Yixiao Chen
• Ye Ding
• Weinan E
• Jiequn Han
• Li'ang Huang
• Weile Jia
• Zeyu Li
• Ziyao Li
• Yinnian Lin
• Yihao Liu
• Xinzijian Liu
• Denghui Lu
• Marián Rynik
• Shaochen Shi
• Ping Tuo
• Bo Wang
• Haidi Wang
• Han Wang
• Yingze Wang
• Yu Xia
• Fengbo Yuan
• Jiabin Yang
• Haotian Ye
• Jinzhe Zeng
• Duo Zhang
• Linfeng Zhang
• Yuzhi Zhang

New features:

• parallel training (#892 #905 #913) (Bytedance)
• automatically determine the sel from the training data. (#831)
• build low and high precision at the same time (#879)

Performance improvement:

• speedup tabulate cuda kernel by reducing shm using (#830) (Bytedance)
• speedup format_nlist_b (#832 #845)

Enhancements:

• support to specify CUDA/ROCm root in python pkg building (#834) (Bytedance)
• use cached Session to speed up py tests (#833)
• add message for DecodeError raised when using model compression (#839)
• remove cub include for CUDA>=11 (#866)
• Add Errcheck after every kernel function runs And merge redundant code (#855)
• adapt changes to auditwheel directory in manylinux (#889)
• enhance the cli to generate doc json file (#891)
• raise warning before training if sel is not enough (#914)

Bug fixings:

• fix bug 824 and Synchronize updates to CUDA cod (#828)
• Fix the empty neighbor distance array in neighbor_stat.py (#882)
• fix InvalidArgumentError caused by zero sel and optimize zero matrix (#900)
• fix 'NoneType' has no len() in auto_sel (#911)

New feature:

• derivatives for deep tensor (#805)

Performance improvement:

• speedup ROCm kernels which use atomicAdd (#809 #815 ) (from ByteDance)
• speedup CUDA kernels (use atomicAdd inside) by reducing the global memory write (#811)

Enhancement:

• add type-embedding developer doc (#762)
• add model compression support for models with exclude_types feature (#754)
• improve the doc and user interface of model compression (#772)
• allow c++ tests to run without internet (#785)
• support converting models generated in v1.3 to 2.0 compatibility (#725)
• give a default value to T and convert models from v1.2 to 2.0 compatibility (#789)
• improved documents for conda (#798)
• throw a message if tf runtime is incompatible (#797)
• capture OOM and print debug message (#801)

Bug fixings

• fix bug of custom op's multiple initialization (#812)
• fix bug of empty input in gelu.cu (#800)
• fix model compression bug of zero environment matrix (#808)
• box.npy is not necessary for nopbc (#810)
• fill rij with zero (#818)
• NOT_LOADABLE should be tuple (#825)

New features:

• Add subcommand and python interface to calculate model-deviation (#715)

Enhancements

• Use fmod to wrap the coord of atoms. UT for force/virial ops (#741)
• UT for model devi C++ interface (#731)
• add CUDA/ROCM buidling documents (#739)
• add op unittests for prod_force, prod_virial, prod_force_grad and prod_virial_grad (#703)

Bug fixings:

• fix bug of adding int to a None random seed (#705)
• reuse the zero layer rather than building a new one (#714)
• fix CI (#739)

New hardware support:

• ROCM support

Enhancement:

• Document and examples for the C++ interface
• Instructions for the i-pi
• Document for the network size and sel

Bug fixing:

• Illegal nlist #680
• Bug in prod_virial_grad that causes wrong results when training with virials #685
• Uniform random seed #691

Increment to v2.0.0-alpha:

New features:

• Atom type embedding
• Model deviation for virial

Enhancement:

• Improved documentation
• Better support for dipole and polarizability learning
• bit operations to encode neighbor information
• MPI support for atomic model deviation #628
• UT for GPU code #569
• UT for model compression #586
• Test Lammps build #600

Bug fixings

• cuda memory access error #566
• relative force model deviation is not copied back at single precision #599
• correct way of allocating memory in float precision #612
• fix TB logdir remove bug #617
• Append out_file when lammps restarts #640

What's new to v2.0.0-alpha.0

• Training and inference the dipole (vector).
• Split of training and validation dataset.

Enhancement:

• Strict argument check in the input script.
• Update readme for v2.0
• Auto conversion of input file to v2.0 compatibility

Bug fixings:

• Fix bugs of broken examples.

The very first alpha release of deepmd-kit version 2.0.0. It includes the following new features

• Model compression
• New descriptor: three body embedding
• Hybridization of descriptors
• Long-range modification
• Type embedding (under development)
• Training and inference the dipole (vector) and polarizability (matrix). (under development)
• Split of training and validation dataset. (under development)
• ROCm device support (under development)

Enhancements

• More efficient training: all customized OPs are implemented with GPU.
• Parallel training with multiple GPU support (under development)

Improvement of the code for developers

• Supports version of the model. Easily check model compatability
• Clear and pythonic python interface
• C++ API that can be tested independently
• OP supports multi-device.

Bug fixings:

• remove using namespace std. Solves compiling compatability problem.
• added deepmd namespace for the C++ API

Bug fixing:

• Fix lammps memory leak when initialized pair_style deepmd for multiple times. #392
• Fix GPU memory issues. #393 #407 #424

Improvements:

• Compiling with Cuda 11.0 and 11.1 #390

Bug fixings:

• Fix stress tensor bug of ASE interface #338
• Fix neighbor list bug that may cause inconsistent results between CPU and GPU #334

Bug fixings

• Fix several compiling issues of v1.2.3

New feature:

• added tool to convert a v1.2 compatible model to v1.3 compatible model.

Bug fixing:

• neighbor list bug in the GPU implementation.

Bug fixing:

• Compulsory label requirement if the corresponding prefactor is set to non-zero. The current behavior is when then label is missing, the corresponding term does not appear in the loss function
• Optional requirement of loss in dipole and polarizability training

Improvement:

• Recommend consistent TensorFlow versions for python and C++ interfaces.

New features:

• Support input script in yaml format #271
• Enable setting test size individually for each system #267

Improvements:

• Support building with TF2.3
• Improved documentation. Automatically generated doc page. Explained keys in the training input script. #297
• Compile with latest stable release of lammps (stable_29Oct2020). Old releases are not supported anymore. #302
• Multi-device and multi-implementation support for OPs. #254

Bug fixings:

• Allow ntypes_model > ntypes_data (fix #261) #296
• Bug in setting coeff_ener when coeff_atom_ener is given #290

New features:

• fast gelu implementation
• relax the restriction on the maximal number of neighbors to 4096
• one-sided embedding net

Bug fixings:

• check if the cell volume is positive.
• bugs of dplr
• loading coord when data has only one frame
• ase calculator None-PBC system
• details printing when testing on multiple systems.
• bugs in transform

Bug fixing:

• GPU implementation: stop the program when the number of neighbors is larger than the limit.

New features of dp train:

• Polarizability and dipole fitting
• If provided with stop_lr, the decay_rate will be computed automatically
• Support non-pbc system: add an empty file named nopbc to the data system.
• Use envs TF_INTRA_OP_PARALLELISM_THREADS and TF_INTER_OP_PARALLELISM_THREADS to control the multi-threading of tf, clean up command line options.
• When the key systems is provided with a string, all possible systems will be recursively searched within the dir given by systems
• User specific atomic energy.
• Exclude types when building descriptors.
• User specific activation function and network precision.
• Transform neural network parameters.

New features of dp test:

• Recursively search all possible systems in a directory.
• Weighted average of test results

Bug fixings:

• Cuda version compatibility
• raw_to_set when nframe == 1
• Slow test when testing multiple systems
• Potentially wrong neighbor list when number of neighbors is larger than 256.
• Fix bug #163 : broken lib link of dp_ipi
• Infering number of types from data

Big fixings:

• KeyError in dp test (#165 )
• Protect the std calculation