PyTorch misc
Collection of code snippets I've written for the PyTorch discussion board.
All scripts were testes using the PyTorch 1.0 preview and torchvision 0.2.1
.
Additional libraries, e.g. numpy
or pandas
, are used in a few scripts.
Some scripts might be a good starter to create a tutorial.
Overview
-
accumulate_gradients - Comparison of accumulated gradients/losses to vanilla batch update.
-
adaptive_batchnorm- Adaptive BN implementation using two additional parameters:
out = a * x + b * bn(x)
.
-
adaptive_pooling_torchvision - Example of using adaptive pooling layers in pretrained models to use different spatial input shapes.
-
batch_norm_manual - Comparison of PyTorch BatchNorm layers and a manual calculation.
-
change_crop_in_dataset - Change the image crop size on the fly using a Dataset.
-
channel_to_patches - Permute image data so that channel values of each pixel are flattened to an image patch around the pixel.
-
conv_rnn - Combines a 3DCNN with an RNN; uses windowed frames as inputs.
-
csv_chunk_read - Provide data chunks from continuous .csv file.
-
densenet_forwardhook - Use forward hooks to get intermediate activations from
densenet121
. Uses separate modules to process these activations further.
-
edge_weighting_segmentation - Apply weighting to edges for a segmentation task.
-
image_rotation_with_matrix - Rotate an image given an angle using 1.) a nested loop and 2.) a rotation matrix and mesh grid.
-
LocallyConnected2d - Implementation of a locally connected 2d layer.
-
mnist_autoencoder - Simple autoencoder for MNIST data. Includes visualizations of output images, intermediate activations and conv kernels.
-
mnist_permuted - MNIST training using permuted pixel locations.
-
model_sharding_data_parallel - Model sharding with
DataParallel
using 2 pairs of 2 GPUs.
-
momentum_update_nograd - Script to see how parameters are updated when an optimizer is used with momentum/running estimates, even if gradients are zero.
-
pytorch_redis - Script to demonstrate the loading data from redis using a PyTorch Dataset and DataLoader.
-
shared_array - Script to demonstrate the usage of shared arrays using multiple workers.
-
shared_dict - Script to demonstrate the usage of shared dicts using multiple workers.
-
unet_demo - Simple UNet demo.
-
weighted_sampling - Usage of WeightedRandomSampler using an imbalanced dataset with class imbalance 99 to 1.
Feedback is very welcome!