PyTorch-like deep learning library with dynamic computational graph construction and automatic differentiation
MIT License
PyTorch-like deep learning library with dynamic computational graph construction and automatic differentiation. Automatic differentiation is based on Autograd
written by Dougal Maclaurin, David Duvenaud and Matt Johnson.
In order to create your own model you have to inherit from Module class and override forward
method. Later on, you can train your model or use it as part of a bigger network.
from beacon.nn import beacon
from beacon.nn import Module, Linear
from beacon.functional import functions as F
class MyModel(Module):
def __init__(self):
super().__init__()
self.fc1 = Linear(inputs=2, outputs=4)
self.fc2 = Linear(inputs=4, outputs=4)
self.fc3 = Linear(inputs=4, outputs=1)
def forward(self, x):
x = F.sigmoid(self.fc1(x))
x = F.sigmoid(self.fc2(x))
x = F.sigmoid(self.fc3(x))
return x
model = MyModel()
When creating a model it is possible to customize the way that parameters are initialized. For example:
from beacon.nn.init import xavier_normal, zeros
self.fc2 = Linear(inputs=4, outputs=4, weight_initializer=xavier_normal, bias_initializer=zeros)
An easier way to get started is to use a predefined sequential model.
from beacon.nn.models import Sequential
from beacon.nn.activations import Sigmoid
model = Sequential(
Linear(2, 4),
Sigmoid(),
Linear(4, 4),
Sigmoid(),
Linear(4,1),
Sigmoid()
)
from beacon.data import data_sequence, data_generator
x_train = [[1, 0], [0, 0], [0, 1], [0, 0]]
y_train = [[1], [0], [1], [0]]
There are two possible ways of preparing data:
X, Y = data_sequence(x_train, y_train, batch_size=4, shuffle=True)
for x, y in data_generator(x_train, y_train, batch_size=4, shuffle=True):
do_some_task(x, y)
from beacon.optim import Adam
optimizer = Adam(model.parameters(), lr=0.1)
model.train()
for epoch in range(1, 1001):
full_loss = 0
n_loss = 0
for x, y in data_generator(x_train, y_train, batch_size=4, shuffle=True):
optimizer.zero_grad()
output = model(x)
loss = F.mean_squared_error(output, y)
loss.backward()
optimizer.step(epoch)
full_loss += loss.item()
n_loss += 1
print(f"Epoch: {epoch}, Loss={full_loss/n_loss}")
model.eval()
with beacon.no_grad():
for x in x_test:
output = model(x)
print(output)
print(output.argmax())
Beacon can be easily extended by defining your own custom activation functions, loss functions or optimizers.
When defining your activation function you have to make sure that all operations are differentiable. Your activation function has to take tensor as a parameter and return a tensor. When you define your activation function it can be used just as predefined activations.
from beacon.tensor import functions as fn
from beacon.tensor import Tensor
def my_activation(t: Tensor) -> Tensor:
return fn.sin(t)
If you want to be able to use your custom activation function in the sequential model, you have to create a class which inherits from Activation class. When you do that, you can use your custom activation layer just as predefined layers.
from beacon.nn.activations import Activation
class MyActivation(Activation):
def __init__(self):
self.activatino = my_activation
Or if you don't want to define a separate function:
class MyActivation(Activation):
def __init__(self):
self.activation = lambda x: fn.sin(x)
When defining your loss function you have to make sure that all operations are differentiable. Your loss function has to take two tensors as parameters and return 0-dimensional tensor. When you define your loss function it can be used just as predefined losses.
from beacon.tensor import functions as fn
from beacon.tensor import Tensor
def my_loss(output: Tensor, target: Tensor) -> Tensor:
return fn.sum(fn.sin(fn.square(output - target)))
You can add sum over batch size decorator to your custom loss function, for example:
from beacon.functional import functions as F
@F.sum_over_batch_size
def my_new_loss(output: Tensor, target: Tensor) -> Tensor:
return fn.sin(fn.square(output - target)) # there is no fn.sum like in the previous example, it is done automatically
When defining custom optimizer you have to inherit from Optimizer base class and redefine _step
method. When you define your optimizer it can be used just as predefined optimizers.
from beacon.optim.optimizer import Optimizer
from beacon.tensor import functions as fn
class MyOptimizer(Optimizer):
def __init__(self, parameters, lr=0.01):
super().__init__(parameters, lr=lr)
def _step(self, epoch):
for parameter in self.parameters:
parameter -= fn.square(self.lr*parameter.grad)