ModelRefs / PyTorch Basics — Tutorial

PyTorch Basics — Tutorial

Tensors, autograd, and nn.Module — the three building blocks of deep learning in PyTorch. Covers Tensors — NumPy arrays with GPU support.

Overview

Tensors, autograd, and nn.Module — the three building blocks of deep learning in PyTorch

Level: Intermediate. Estimated reading time: 40 minutes.

Tensors — NumPy arrays with GPU support

PyTorch tensors are the core data structure. They behave like NumPy arrays but can live on GPU and track gradients. Key differences from NumPy: (1) .to('cuda') moves a tensor to GPU; (2) requires_grad=True enables gradient tracking; (3) operations return new tensors (immutable by default).

Shape is everything: check tensor.shape constantly. The convention is (batch, channels, height, width) for images, (batch, sequence, features) for text.

nn.Module — the building block of models

Every neural network in PyTorch inherits from nn.Module. You define the architecture in __init__ and the forward pass in forward(). PyTorch's autograd traces through forward() to build the computational graph for backpropagation.

Key methods: model.parameters() — all learnable tensors. model.eval() — switch off dropout/batchnorm training modes. model.to(device) — move all parameters to CPU/GPU. model.state_dict() — serialisable snapshot for saving.

The training loop pattern

A standard PyTorch training loop has five steps per batch: 1. optimizer.zero_grad() — clear old gradients 2. output = model(X) — forward pass 3. loss = criterion(output, y) — compute loss 4. loss.backward() — compute gradients 5. optimizer.step() — update parameters

Wrap evaluation in with torch.no_grad() to skip gradient computation. Move model and data to the same device or you'll get a device mismatch error.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to PyTorch Basics — Tutorial.