Why Use Batch Size

In deep learning, batching is an essential concept for efficiently training models, especially when working with large datasets. This article will explain the use of batch processing in PyTorch and how to work with the nn.Linear module.

1. What is Batch Processing?

Batch processing refers to the practice of processing multiple input samples at once, rather than one at a time. This is important for optimizing both training and inference, especially when working with powerful hardware like GPUs.

• Batch Size (B): The number of samples in a batch. For example, a batch size of 10 means you are processing 10 samples simultaneously.
• Input Shape (B × Nin): If you have a batch of inputs, the shape will be [B, Nin], where B is the batch size, and Nin is the number of input features per sample.

2. Why Use Batch Size?

Using batch processing comes with several advantages:

• Efficiency: GPUs are optimized for parallel processing. By using batches, you make use of the full computational power of the GPU, speeding up training and inference.
• Better Statistics: Advanced models may compute statistics (e.g., mean and variance) over the batch. Larger batch sizes tend to give more accurate statistics, improving model performance.
• Faster Convergence: Optimizers like stochastic gradient descent (SGD) use the average gradient over the batch to update the model weights, reducing noise and helping the model converge faster.

3. Example with nn.Linear and Batching

Let's explore how batch processing works with nn.Linear in PyTorch. In this example, we will process a batch of 10 samples.

import torch
import torch.nn as nn

# Create a linear model: input feature size 1, output feature size 1
linear_model = nn.Linear(1, 1)

# Create a batch of inputs, size (10, 1)
x = torch.ones(10, 1)

# Pass the batch through the model
output = linear_model(x)
print(output)
        

The input tensor x has a shape of [10, 1], which means we are passing a batch of 10 samples, each with 1 feature.

When we pass this tensor through linear_model, PyTorch processes all 10 inputs simultaneously, leveraging the parallel processing capabilities of the GPU. The output will have the same shape, [10, 1], since we are mapping from 1 input feature to 1 output feature for each sample in the batch.

4. Example with unsqueeze and Reshaping

When working with 1D tensors, such as temperature data, we often need to reshape them to meet the requirements of nn.Linear, which expects inputs to be of the form [B, Nin]. Let's look at how to reshape data using unsqueeze.

# Original 1D tensors
t_c = [0.5, 14.0, 15.0, 28.0, 11.0, 8.0, 3.0, -4.0, 6.0, 13.0, 21.0]
t_u = [35.7, 55.9, 58.2, 81.9, 56.3, 48.9, 33.9, 21.8, 48.4, 60.4, 68.4]

# Convert to tensors and reshape using unsqueeze
t_c = torch.tensor(t_c).unsqueeze(1)  # Reshape to [11, 1]
t_u = torch.tensor(t_u).unsqueeze(1)  # Reshape to [11, 1]

# Check the shape
print(t_c.shape)  # Output: torch.Size([11, 1])
        

The unsqueeze(1) method adds an extra dimension to each tensor, transforming them from 1D tensors of shape [11] into 2D tensors of shape [11, 1].

This reshaping is necessary because nn.Linear expects a 2D input with the shape [B, Nin], where B is the batch size (11 in this case) and Nin is the number of input features per sample (1 here).

5. Batch of Images Example

In the case of image data, the input tensor typically has the shape [B, C, H, W], where:

• B: Batch size (number of images)
• C: Number of channels (3 for RGB images)
• H: Height of the image
• W: Width of the image

For example, if we have 3 RGB images of size 64x64 pixels, the input tensor would have the shape [3, 3, 64, 64]. This allows us to process a batch of images at once.

6. Summary

• Batch Processing: Allows multiple samples to be processed simultaneously, making full use of GPU resources for faster computation.
• Reshaping Input: When using nn.Linear, the input must have the shape [B, Nin], where B is the batch size and Nin is the number of features per sample.
• Efficient Computation: By using batches, GPUs are fully utilized, and models can train and infer much faster than processing inputs one at a time.

Batch processing is a crucial concept for training and deploying machine learning models efficiently, and PyTorch provides the necessary tools to handle batched inputs easily. Understanding how to reshape data and utilize batching properly will help you make the most of your models, especially when working with large datasets and GPUs.

Further Reading

1.