This is a minor release. See https://github.com/pfnet/chainer/milestone/28?closed=1 for the complete list of solved issues and merged PRs.

New features

• Parameter shape placeholder is added (#1363).
  • Linear and Convolution2D links now support automatic initialization at the first forward propagation (similar to shape inference in other frameworks).
  • You can pass None as the input (channel) size to these links, then they automatically allocate and initialize the parameters based on the actual input size at the first forward propagation.
• New Functions/Links:
  • crf1d: Linear chain CRF layer (#1092)
  • separate: Get a tuple of all “rows” along an axis, inverse function of stack (#1345)
  • StatefulPeepholeLSTM: LSTM layer with peephole connection (#1207)
• New CuPy functions
  • fill_diagonal: Fill diagonal elements by constant (#1404)
• CuPy scan performance is improved (#1367, thanks @kikusu!)

Bug fixes

• cupy.ndarray.view: fixed dtype (#1471)
• get_cifar10, get_cifar100: fixed paths and cache (#1449, thanks @owruby!)
• get_cifar10, get_cifar100: decoding error on Py3 (#1466, thanks @Fohte!)
• ImageDataset: fixed for gray-scale images (#1445)
• LogReport: failed on Windows (#1394, #1412, thanks @danpansa for reporting it!)
• ParallelUpdater: fixed for dict mini-batch (#1464)
• SerialIterator: did not shuffle if dataset size is divisible by batchsize (#1416, #1421)

Improvements

• Chain.zerograds: removed quadratically-duplicated internal zerograd calls (#1434)
• Unnecessary copies of arrays are removed (#1444)

Also thank you @uchida (#1440) and @henry0312 (#1463) for the documentation improvements!

This is a minor release. See https://github.com/pfnet/chainer/milestone/26?closed=1 for the complete list of solved issues and merged PRs.

Summary:

• word2vec example is improved (#641)
• check_backward: Support dtype option. It tests backward propagation of target function with specified dtype (#1247)
• Added GradientHardClipping hook for optimizers (#1315, thanks @tohmae!)
• Variable’s initializer now accepts grad option to initialize the gradient, and pickle of Variable now serializes the gradient as well as data (#1298, thanks @hitottiez!)
• New/improved Functions
  • rollaxis (#1401)
  • getitem: Support integer indexing and Ellipsis (#1343)
  • transpose_sequence: Made 10x faster (#1376)
  • max_pooling_2d: CPU implementation is made much faster (#1334, thanks @tkng!)
• Improved Links
  • LSTM: set_state method is added (#1381, thanks @fukatani!)
• Bug fixes
  • LogReport file name format is fixed (#1398, thanks @dsanno!)
  • Support cupy.dot along zero-sized axis (#1374)
  • Fix async to_cpu (#1414)

This is a minor release that contains the following updates:

feature #914 #1142 #1168 #1204 #1217 #1285 #1286 #1289 #1329 #1330 #1331 #1358 #1359 #1373
other #1357
enhancement #886 #1112 #1302 #1311 #1312 #1342
test #1302
document #1328 #1341 #1354 #1355 #1371
bug #1297 #1336 #1341 #1346 #1352 #1355 #1364

Summary:

• Chainer now supports dataset and training loop abstraction (#1285).
  • It contains the following components.
    • Dataset and Iterator to extract mini batches by iterating over datasets
    • Trainer, Updater, and Extension to customize the training loop with low cost
    • Reporter to collect statistics from inside of the models
  • It also contains off-the-shelf support of data parallel learning with multiple GPUs (#1358, thanks @amitibo!!).
  • MNIST, PTB, and ImageNet examples are updated. They are now using Trainer.
  • Tutorial is also updated to support the updated examples.
• New links and Functions:
  • CReLU (#1142)
  • Gradient noise optimizer hook (#1168, thanks @kashif!!)
  • Argmin/argmax (#1217)
  • Triplet loss (#1330, thanks @amitibo!!)
  • Normalize L2 (#1331, thanks @amitibo!!)
• Enhancement of links and Functions:
  • BatchNormalization supports flexible initialzation (#1289, thanks @fukutani!!).
  • Classifier is improved. It can change the function to calculate accuracy (#1286) and can accept more than two Variables on call (#1204).
• Asynchronous data transfer using a CUDA stream is now supported (#1112).

This is a minor release that contains the following updates:

feature #1045 #1068 #1172 #1189 #1194 #1195 #1216 #1220 #1221 #1246 #1250 #1251 #1252 #1256 #1257 #1262 #1265 #1266 #1275 #1276
contribution-welcome #1152
other #1301 #1303 #1313
enhancement #1078 #1268 #1291 #1293 #1324 #1325
test #1246 #1287 #1299
document #1272 #1284 #1296 #1306 #1307 #1314 #1316 #1319
bug #1152 #1278 #1282 #1307 #1323

Summary:

• Chainer Functions/Links
  • New Functions
    • bias, scale: elementwise product/summation with broadcasting #1252
    • logsumexp: stable computation of log-sum-exp #1216
    • minimum: elementwise minimum operation #1250
    • permutate: permutes array in specified order along an axis #1194
    • transpose_sequence: transposes a list of arrays #1195
  • New Links
    • Bias, Scale: elementwise product/summation with learnable parameters. # 1252
  • Updates
    • Functions can now directly accept numpy.ndarray or cupy.ndarray as well as chainer.Variable. Function automatically wraps arrays with Variable in those cases. #1221
    • NegativeSampling: adds ignore_label attribute that specifies data points to be ignored. #1045 #1275 (thanks @cemoody!)
    • Variable.__getitem__: support newaxis #1257
    • sum: fixed backward of 0-dimensional sum #1278 (thanks @fukatani!)
• CuPy functions
  • New functions
    • flatnonzero, ix_, nonzero #1172 (thanks @kikusu!)
    • stack #1276
  • Updates
    • concat: supports negative axis argument #1266
• New Optimizers
  • SMORMS3: #1068 (thanks @benob!)
• Other updates
  • gradient_check.assert_allclose is moved to chainer.testing.assert_allclose, the former is now deprecated. #1246
  • gradient_check.check_backward: adds no_grad option that specifies variables that are skipped. (thanks @wkentaro!) #1256
  • cuda.get_device: it now ignores NumPy scalar inputs #1268
  • Contribution guide is updated to follow the new release cycle #1272

This is a revision release that contains the following updates:

bug #1219 #1263
enhancement #1240 #1248 #1259
install #1153 #1214
test #1237 #1245 #1255
document #1238 #1244
example #1267

This is a small release that contains some improvements and bugfixes (thanks @mitmul, @kcarnold, @wkentaro!).

Announcement: We will change the release cycle. We currently have two revision releases followed by one minor release that takes two weeks each. Starting from the next release, we allow new features added in any scheduled releases. When the release includes new features, the minor version is incremented. We are preparing updates of the contribution guide.

This is a minor release that contains the following updates:

feature #519 #820 #911 #926 #930 #939 #966 #967 #980 #1028 #1086 #1113 #1125 #1126 #1136 #1154 #1157 #1158 #1160 #1162 #1163 #1164 #1165 #1166 #1169 #1179 #1190 #1191 #1203 #1212 #1218 #1222 #1230 #1234
bug #1192 #1193 #1196 #1198 #1199 #1208 #1226
enhancement #1176 #1177 #1215
test #1140 #1186 #1201 #1210 #1211 #1213 #1229 #1233
document #1182 #1187 #1188 #1200 #1206
example #1234

Summary (many thanks to the contributions!)

• Some important features are added
  • Caffe support is improved, and we now support ResNet models https://github.com/KaimingHe/deep-residual-networks #1234
  • Weight initializer: a new standard way to initialize parameters of links #1086 (thanks @EdwardRaff!)
  • Profiling API: an API to manage CUDA profliing utilities #967
  • Loosen type condition: many functions now support float16 and float64 variables #1136, #1162, #1163, #1164, #1165, #1166, #1203
• New Functions
  • cast: Cast the data type of arrays #1230
  • convolution_2d: Support cover_all option #926 (thanks @wkentaro!)
  • __getitem__, get_item: Differentiable basic indexing (including cropping) #1154 (thanks @wkentaro!)
  • hard_sigmoid: Hard sigmoid activation function #1125
  • log1p, expm1: Elementwise stable log(1+x) and exp(x)-1 #1222 (thanks @yuji-mizobuchi!)
  • linear_interpolate: Linear interpolate function, that calculates px + (1-p)y #1126
  • roi_pooling_2d: Max pooling over specified regions of interest #939 (thanks @wkentaro!)
• Updated Functions
  • accuracy: ignore_label option is added #1113 (thanks @henry0312)
  • copy: array copy between CPU and GPU is supported
• New Links
  • StatelessLSTM: LSTM link for users that want to manage the recurrent state by themselves #980
• Others
  • Computational graph is improved #1169 #1218 (thanks @wkentaro!)
  • Workspace size of cuDNN convolution functions are now customizable via chainer.cuda.set_max_workspace_size #1179

The next minor release is planned on July 12.

This is a revision release that contains the following updates:

test #1141
example #1004
bug #1161 #1170 #1175 #1178
enhancement #1145
document #1155

This release includes some bug fixes. For example, huber_loss is fixed to correctly handle the case that the loss output is further fed to other Functions.

We apologize that we forgot to notify that the release was delayed for one week (due to the holidays in Japan). The next release is planned on May 31, and is a minor release with some new features.

This is a revision release that contains the following updates:

test #1055 #1114 #1128 #1132 #1134
enhancement #1093 #1111 #1122 #1127 #1130
document #1116 #1117 #1120 #1121 #1137
example #1144
bug #1129 #1135

Summary

• Chainer now runs with cuDNN v5 RC.
• Made the workspace size of cuDNN convolution configurable. Note that this API is undocumented yet and will be officially supported in the next minor release. We also revised the default workspace size, which will probably improve convolution2d in some network architectures.
• Improved the memory pool efficiency by rounding up the CUDA allocation size to multipliers of 256 bytes.
• Fixed cuda.to_cpu to support copy from non-current devices.

This is a minor release that contains the following updates:

feature #762 #832 #847 #932 #940 #970 #973 #974 #978 #1002 #1007 #1067 #1069 #1104
example #806 #1014 #1046
enhancement #1013 #1048 #1107
install #1039 #1048 #1094 #1095
test #1084 #1096 #1100
document #1089
bug #1037 #1091 #1095 #1097 #1098 #1106 #1108

Summary

• CaffeFunction is now supported for Python3 (using protobuf 3).
• Start support of NumPy 1.11
• New/updated Functions
  • clip: Differentiable version of numpy.clip
  • embed_id: ignore_label option is added
  • huber_loss: Generalized smooth L1 loss (thanks @wkentaro!)
  • softmax_cross_entropy: cache_score option is added
  • split_axis: force_tuple option is added
• New/updated Links
  • BatchNormalization: Fixed errors on running with multiple GPUs, and also fixed a critical bug of finetuning mode
  • EmbedID: ignore_label option is added
• Lasso: optimizer hook for L1 regularization (thanks @Kiikurage!)
• PrintHook (a function hook) is added for debug printing
• New examples
  • VAE example (thanks @ywatanabex!)
  • Text generation example (thanks @hamatz!)

This is a revision release that contains the following updates:

example #1032 #1057 #1064 #1071
enhancement #904 #1047 #1052 #1061 #1063 #1065 #1066 #1081
install #960 #962 #1051
test #1040 #1042 #1053 #1060 #1072 #1077
document #946 #1008 #1035 #1038 #1043 #1044 #1049 #1054 #1056 #1059 #1072 #1074 #1075 #1077 #1079
bug #1031 #1033 #1050 #1058 #1070

Summary

• We made many documentation improvements in this release. Check the latest documentation at http://docs.chainer.org/
• Fixed HDF5Deserializer to deserialize an object from an HDF5 group (thanks @jnory!)
• Fixed CuPy for slicing by negative steps

This is a revision release that contains the following updates:

bug #998 #999 #1000 #1011 #1016 #1022 #1026
enhancement #942 #1012
install #997 #1009 #1021
test #994 #998 #1001 #1023
document #704 #1005 #1015 #1017 #1019 #1024 #1025
example #1010

Some bugs are fixed in this release.

NOTE: We found that some features of Chainer do not work correctly on Python 3.5.0 but do work on Python 3.5.1 (see #997). We updated the document and CI. We highly recommend you to use 3.5.1 instead of 3.5.0.

This is a minor release that contains the following updates:

feature #402 #457 #518 #597 #599 #613 #689 #756 #786 #830 #842 #888 #899 #921 #923 #933 #943 #985 #986 #987 #989 #991
bug #972 #977 #979 #990
enhancement #278 #302 #882 #884 #898 #934 #935
test #963 #981 #988
document #956 #957 #959 #965 #968 #969 #976 #993
example #961 #983 #984

Summary

• Core functionalities
  • Debug mode is introduced. In debug mode, many values are verified during computations (e.g. NaN during forward/backward computations, the range of label integers for softmax_cross_entorpy, etc.). Call set_debug(True) to enable the debug mode. Note that debug_mode introduces computational overheads.
  • FunctionHook is added. Users can freely adds hook functions to every Function forward/backward computation. For example, TimerHook measures the timing of forward/backward computations to diagnose performance issues.
  • cuDNN v4 is now officially supported (thanks @kashif!).
  • Variable.debug_print method is added (thanks @sinhrks!). This method returns a string containing information of the variable (e.g. device ID, shape and dtype, statistics of data and grad, etc.).
• New Functions
  • det, batch_det (matrix determinant, thanks @cemoody!)
  • log_softmax
  • slstm (LSTM along binary trees)
  • unpooling_2d (spatial upsampling, thanks @wkentaro!)
• Improved Functions
  • binary_accuracy (added ignore_label option, thanks @t-abe!)
  • ctc_padding (now minibatch can contain sequences of different lengths, thanks @jnishi!)
  • sigmoid_cross_entropy (performance improved)
• New CuPy Functions
  • asfortranarray
• Others
  • Device-to-device copy now leverages GPUDirect if available.

We found that forward computations do not work correctly on Python 3.5.0. We are investigating the error, but we recommend to update to Python 3.5.1, on which forward computations correctly work.

This is a revision release that contains the following updates:

bug #844 #925
enhancement #881 #910 #912 #916 #917 #944
install #950
test #718 #741 #844 #894 #913 #922 #924 #927
document #889 #920 #929 #936 #941
example #922

There are some enhancements added in this release, e.g. slight reduction of memory usage on backward, more readable nvcc error messages, and so on.

This is a revision release that contains the following updates:

test #718 #872
document #829 #874 #883 #892 #902 #906
bug #793 #816 #875 #876 #877 #878 #880 #885 #890 #895 #903 #906 #907 #908
install #873 #880 #893
enhancement #840 #843 #871 #887 #891 #900 #901

Summary

• API Compatibility Policy is now released at http://docs.chainer.org/en/latest/compatibility.html in the official documentation.
• convolution_2d and deconvolution_2d now accept non-contiguous inputs in cuDNN-enabled mode (by copying inputs to contiguous arrays if necessary).
• GPU implementations of some functions are improved (thanks @hrantzsch!).
• Some bugs are fixed.
  • In particular, NPZ serializer now correctly deserializes optimizers.

This is a minor release that contains the following updates:

feature #439 #532 #597 #608 #642 #644 #690 #694 #700 #716 #728 #747 #754 #757 #774 #781 #782 #785 #789 #790 #794 #798 #802 #810 #817 #826 #827 #828 #849 #856
bug #748 #749 #750 #751 #759 #761 #764 #768 #778 #787 #797 #800 #805 #808 #811 #812 #818 #819 #822 #833 #834 #851 #852 #861 #864
other #772
enhancement #527 #752 #755 #769 #791 #792 #795 #831 #838 #845 #846 #848 #866 #867
install #773 #783 #795
test #752 #753 #758 #760 #764 #765 #771 #776 #801 #802 #803 #821 #834 #848 #857 #858 #859 #860 #863 #865
document #714 #779 #796 #799 #813 #823 #836 #854 #862 #870
example #808 #862

Summary

• We have started to develop with our new compatibility policy (#829). It will be put into the official documentation in the near future.
• HDF5 support is now made optional. Users can use Chainer without installing HDF5 and the h5py package. For environments without HDF5, NumPy NPZ serialization is added.
• NumPy 1.10 is now supported.
• Chainer now runs with cuDNN v4 RC1, though does not leverage its full capability, yet (as well as cuDNN v3).
• This release also contains many improvements on differentiable functions, links, and CuPy functions. Here are lists of additional features:
  • New Chainer functions:
    • batch_l2_norm_squared (#828): Thanks @hrantzsch!
    • binary_accuracy (#757): Accuracy for binary classification. Thanks @t-abe!
    • broadcast_to (#826)
    • elu (#754): Thanks @muupan!
    • expand_dims (#690)
    • inv (#608): Differentiable matrix inversion. Thanks @cemoody!
    • maxout (#856)
  • New links:
    • StatefulGRU (#817): GRU with LSTM chain-like interface
    • Maxout (#856)
  • New CuPy functions:
    • broadcast_to (#644)
    • roll (#827)
  • Enhancement of existing functions:
    • fixed_batch_normalization (#798): Backward operation is now supported
    • sigmoid_cross_entropy (#790): ignore_label is supported
• This release also contains other improvements on MemoryPool (#785, thanks @jnory!) and testing utilities (#782).

We start a new minor release cycle from this release: one minor release for every six weeks. Revision release cycle is not changed from biweek updates.

This is a revision release containing following changes:

bug #684 #708 #723 #725 #730 #732 #733 #736 #737
other #727 #738 #744
enhancement #584 #706 #712 #719 #721 #722 #739
test #692 #693 #709
document #619 #688 #710 #713 #717 #735 #743
example #726

Summary

• Installation issue is fixed.
• Installation guide is added. http://docs.chainer.org/en/stable/install.html
• Most of them are bug fixes and test improvements.

This is a minor release containing following changes:

feature #92 #181 #306 #315 #345 #363 #399 #419 #546 #553 #558 #573 #576 #587 #589 #591 #603 #604 #605 #636 #645 #658 #661
example #473 #624
other #630 #646 #647 #649 #668
enhancement #620 #621 #643 #663 #664 #673
test #70 #618 #626 #640 #648 #653 #654 #657
document #325 #629 #631 #633 #637 #669 #671 #674
bug #628 #632 #634 #635 #639 #651 #655 #660 #662 #665 #672

This release includes some big changes on Chainer and CuPy. In particular, the installation process is updated. See README.md for details.

Summary of updates on Chainer:

• #363, #573: Link/Chain is introduced. These replace FunctionSet and are the new standard ways to write one’s own models. We also support their serialization in HDF5 format. See the tutorial and reference for details.
• Function is not allowed to hold parameters to optimize from this version. Write a pure function and wrap it by a new subclass of Link to combine parameters with functions.
• “Parameterized functions” of previous versions are now moved to chainer.links namespace as links and chains. We still leave aliases at chainer.functions for backward compatibility.
• New links and chains:
  • LSTM: fully-connected long short-term memory layer as a chain
  • #419 GRU: gated recurrent unit
  • MLPConvolution2D: mlpconv layer of Network-in-Network
  • Classifier: simple multi-class classifier based on an arbitrary predictor network
  • #315, 591, #645 Deconvolution2D: reverse propagation of Convolution2D by @tnarihi, @t-abe, and @wkentaro
• New functions:
  • #546 hinge: hinge loss function by @mitmul
  • #553 contrastive: contrastive loss function for siamese networks by @mitmul
  • #558 min, max: differentiable amin/amax by @tscohen
• Improved functions:
  • #621 transpose: default transposition (w/o axes specified) by @kiyukuta
• Improved links/chains:
  • #589 EmbedID: multi-dimensional embedding

Summary of updates on CuPy:

• #587: The implementation of CuPy is ported to Cython. This reduces the CPU-side overhead on using CuPy. Further optimization will be done.
• New CuPy APIs:
  • #604 repeat by @wkentaro
  • #605 tile by @wkentaro
  • #636 bincount

This is a revision release that contains following changes:

bug #566 #574 #583 #585 #586 #592 #595 #601
other #582
enhancement #548 #567 #579 #607
test #556 #570 #577 #578 #580 #581 #590 #594
document #609
example #598

Performance issue of reduction kernels is fixed. Python3.5 is now supported. We are planning a minor release (v1.5.0) on Nov. 25.

This is a minor release containing following changes:

feature #280 #408 #409 #456 #458 #480 #484 #485 #493 #500 #501 #505 #511 #512 #517 #540 #541 #542 #545
bug #434 #460 #464 #488 #489 #490 #496 #502 #510 #516 #523 #524 #529 #547 #551 #552
other #477 #521
enhancement #467 #476 #478 #491 #506 #549 #554
test #461 #462 #463 #465 #466 #470 #474 #475 #488 #497 #498 #510 #534 #536 #537 #538 #539 #544 #550
document #482
example #530 #535

Summary

• New functions
  • #280 ConnectionistTemporalClassification
  • #480 transpose, swapaxes
  • #485 select_item
  • #512 broadcast
  • #541 where
• Functions spec extensions
  • #500 softmax_cross_entropy with “ignore” label
  • #505 sum along multiple axes
• New optimizers
  • #540 NesterovAG: Netsterov’s accelerated gradient method
• New CuPy APIs
  • #408 random.randint, random.random_integers
  • #493 all/any
  • #517 abs (alias of absolute)
  • #541 where
  • #542 count_nonzero
  • #545 min/max (alias of amin/amax)
• #537, #544 Parameterized test cases are supported both by chainer.testing and cupy.testing
• #535 AlexNet architecture with LRN layers are added to ImageNet example

This time @tscohen gave many improvements and suggestions (thank you!!!). This release also contains many bug fixes. We apologize for the release delay. We continue the biweek updates.

This is a revision release that contains following changes.

test #430 #431 #433 #437 #440 #442 #443 #444 #445 #446 #447 #448 #449 #451
document #420
bug #297 #412 #421 #422 #424 #425 #426 #432 #435 #436 #438 #440 #441 #442 #443 #447

Summary

• Most of them are bug fixes and test improvements.
• We can now pickle CaffeFunction objects (#297).
• Dev team enjoyed the holiday week (a.k.a. “silver week”) in Japan :)