nnabla-ext-cuda

• Format
• Communicator utils: barrier and abort
• Change Dockerfile to Use Build arg instead of generating interactively.

• C++ CLI for CUDA extension
• FP16 usage doc
• Update Dockerfiles and doc for v1
• Write function implementation document.

• Feature/20180622 file format converter
• Workaround cublasGemmStridedBatchedEx
• Follow-up "Re-design graph engine for rewiring"

• Dynamic loss scaling for the mixed precision training
• FFT and IFFT
• ext-cuda build manylinux wheel
• document fix
• Reshape with not inplace option

• Feature/20180613 rename ext cuda wheel
• All-reduce during backward
• Misc build 20180618
• Feature/20180220 cpp training mnist training
• Feature/20180428 pad function
• fix document
• add ceil and floor func to cuda ext
• Fix/20180525 fix build
• Trigonometric Functions
• Big change around build system.
• Change NULL to nullptr
• Correspond to termination of python3 support of futures library.
• Communicator with Half Precision
• [generator] Function templates uses write-only flag in cast
• Function: ClipByNom
• AMSGRAD solver
• Add NOTICE file.
• ClipByValue and ClipGradByValue functions

• [BIG UPDATE] New code geneartor and type specialization (float16 support with TensorCore)
• Hotfix/20180228 out of place division

• [build] install command for libnnabla_cuda.so and headers
• Fix Python wheel target dependencies.

• Data Parallel Distributed Training over Multiple Nodes
• Find Volta
• More efficient workspace memory allocation than cached array
• Add implementation of depthwise convolution.

• Add the Swish activation function
• Fix driver shutting down issue on CUDA
• Add the SELU activation function
• Feature/20171011 out of place allreduce
• LeakyReLU CUDA
• Feature/20170908 quantize
• Fix INQ on Windows
• Feature/20171010 fix build and get coverage
• Add CUDA implementation of INQAffine and INQConvolution
• Remove warning from function generator
• [c++][function] Support broadcasting scalar
• Feature/20170831 separate cpp build
• remove obsolete copyright notice
• Function: matrix_diag and matrix_diag_part
• Function: Batch sample-wise matrix multiplication
• Improve backprop options of batch_normalization with CUDNN

Link: Release note of nnabla v0.9.4

Features

• New functions: AbsoluteError, EpsilonInsensitiveLoss

Changes

• 07ca0a8a3def9db384972bdce4cd91221efdac39 EpsilonInsensitiveLoss
• d262ae6d94edf7b1ed6924ef7e601000b3c72041 AbsoluteError

Link: Release note of nnabla v0.9.3

Features

• b1940ba79a5b944c91d496369ab00bc81e9c9590 NCCL integrated multi-GPU communication for data parallel multi-GPU training working with CIFAR-10/100 examples in nnabla (Requires to build from source with NCCL. No binary package is provided so far. See the installation guide for multi-GPU training)
• 111c37004df9318ce138ee46a5abb317657d163e Nearly 2x faster dilated Convolution by enabling CUDNN 6.0 dilation options.
• fe8db91ab75134e0d39358993c662a97becfdbb9 Rearrange Docker files. A Docker image for tutorials is available at DockerHub nnabla/nnabla-ext-cuda:tutorial. See https://hub.docker.com/r/nnabla/nnabla-ext-cuda/.

Features

• Python 3 support
• New functions: Constant, Rand, Randint, Randn
• Faster Deconvolution with CUDNN: DeconvolutionCudaCudnn

Changes

• bd01bb0a09f68805e42d04f586bdcbacb3525ddc Now it is not required to complete implements.rst.
• 1213f85e2526a10a45180bb55db1eb0b71d59480 Random functions (rand, randint, randn)
• ab77c1a1fa59a5dec3b5fcc634179f9899a8e414 Deconvolution CUDNN implementation
• 243d6ec0677a31b2ef52d39103dfc946b28188e7 Faster host to device transfer
• d45e6f06a68da3be9d49b6e2a1679936db4683f4 Auto revision number generation
• 8430cf2a929b9bd629054b17be2328c08f94e032 Python3.5 support
• 359e420d185fd5e8316e997d65afc4a5659874db Support multiple Python 3 versions
• 17e06432bc9aa6d890c3a33795443783dba07989 Disallow Cython==0.26

Known issues

• If Python exception is raised and not caught until program exiting, it sometimes displays an CUDA error message "driver shutting down". Although it doesn't harm any computation and results, we are investigating this issue further.

CUDA extension of NNabla with NNabla v0.9.1.

• Near all functions and solvers are implemented with our CUDA implementation and CUDNN. See the list of available functions in the CUDA extension.
• Officially supports CUDA 8.0 and CUDNN v6. We previously used CUDA 7, 7.5 and CUDNN v4, 5, 5.1 as well. Building with those probably works well without modification or with little modification.