This is the condensed version of the MNIST training single layer NN with SGD from the Fastai course

Define the path

Here we define the path where all the images can be found. This already contains both training and validation images

path = untar_data(URLs.MNIST_SAMPLE)
Path.BASE_PATH = path

Create dataloader objects

define the paths for sevens and threes
open the images and place them in a tensor
create a stack of all threes and of all sevens
create a training x set including images of threes and sevens
create a training y set of labels of threes and sevens
create the dataset which includes x's and y's
create a dataloader object and define the batch size

threes = (path/'train'/'3').ls().sorted()
three_tensors = [tensor(Image.open(o)) for o in threes]
stacked_threes = torch.stack(three_tensors).float()/255

sevens = (path/'train'/'7').ls().sorted()
seven_tensors = [tensor(Image.open(o)) for o in sevens]
stacked_sevens = torch.stack(seven_tensors).float()/255

train_x = torch.cat([stacked_threes, stacked_sevens]).view(-1, 28*28)
train_y = tensor([1]*len(threes) + [0]*len(sevens)).unsqueeze(1)
dset = list(zip(train_x,train_y))
dl = DataLoader(dset, batch_size=256)

Now do the same thing for the validation data

valid_threes=(path/'valid'/'3').ls().sorted()
valid_three_tensors = [tensor(Image.open(o)) for o in valid_threes]
valid_stacked_threes = torch.stack(valid_three_tensors).float()/255

valid_sevens=(path/'valid'/'7').ls().sorted()
valid_seven_tensors = [tensor(Image.open(o)) for o in valid_sevens]
valid_stacked_sevens = torch.stack(valid_seven_tensors).float()/255

valid_x = torch.cat([valid_stacked_threes, valid_stacked_sevens]).view(-1, 28*28)
valid_y = tensor([1]*len(valid_threes) + [0]*len(valid_sevens)).unsqueeze(1)
valid_dset = list(zip(valid_x,valid_y))
valid_dl = DataLoader(valid_dset, batch_size=256)

Define the initial weights and biases

First define a function init_params which takes in the dimenension, and defines them as differentiable values.
Then create a set of weights (28*28) and biases

def init_params(size, std=1.0): return (torch.randn(size)*std).requires_grad_()
weights = init_params((28*28,1))
bias = init_params(1)
#weights.grad.zero_()
#bias.grad.zero_();

Define functions for training

Batch_accuracy takes in a batch of outputs of the model, pushes them to be between 0 and 1 and compares them with the true answer. The ouput is the average of the right and wrong answers, and therefore isn't the same as the loss function.
validate_epoch for each batch.
linear1 takes in all images and outputs the linear transformation from the weights and biases.
sigmoid
mnist_loss takes in the predictions (outputs of the model) flattens them with the sigmoid and compares them to the targets.
calc_grad takes the predictions, calculates the losses and the gradients using backpropagation. These gradients are attached to all parameters which have been defined previously as differentiable.
train_epoch passes in images, and labels, with the model (linear model in this case) to calc_grad, and updates parameters with grad values. It then runs through all parameters and applies gradient descent using the learning rate.

def batch_accuracy(xb, yb):
    preds = xb.sigmoid()
    correct = (preds>0.5) == yb
    return correct.float().mean()
def validate_epoch(model):
    accs = [batch_accuracy(model(xb), yb) for xb,yb in valid_dl]
    return round(torch.stack(accs).mean().item(), 4)

def linear1(xb): return xb@weights + bias
def sigmoid(x): return 1/(1+torch.exp(-x))
def mnist_loss(predictions, targets):
    predictions = predictions.sigmoid()
    return torch.where(targets==1, 1-predictions, predictions).mean()
def calc_grad(xb, yb, model):
    preds = model(xb)
    loss = mnist_loss(preds, yb)
    loss.backward()
def train_epoch(model, lr, params):
    for xb,yb in dl:
        calc_grad(xb, yb, model)
        for p in params:
            p.data -= p.grad*lr
            p.grad.zero_()

Run for 20 epochs

Define the parameters, and run 20 epochs of train_epoch with a linear model and a learning rate of 1. print out the accuracy using validate_epoch on the model.

lr = 1.
params = weights,bias
for i in range(20):
    train_epoch(linear1, lr, params)
    print(validate_epoch(linear1), end=' ')

Jonathan Shock code blog

Imports

This is the condensed version of the MNIST training single layer NN with SGD from the Fastai course

Define the path

Create dataloader objects

Now do the same thing for the validation data

Define the initial weights and biases

Define functions for training

Run for 20 epochs

Title

An Example Markdown Post

Example Markdown Post

Basic setup

Basic formatting

Lists

Boxes and stuff

Images

Code

Tables

Tweetcards

Footnotes