Binary Cross-Entropy Loss (BCELoss)
Binary Cross-Entropy Loss (BCELoss) is a loss function commonly used in binary classification problems. It measures how well a model’s predicted probabilities match the true binary labels (0 or 1).
Why BCELoss is Important
BCELoss is widely used in tasks such as:
- Binary classification
- Logistic regression
- Discriminator training in Generative Adversarial Networks (GANs)
In GANs, the discriminator decides whether an image is real or fake, making BCELoss a natural choice.
Mathematical Definition
The Binary Cross-Entropy Loss is defined as:
L(y, ลท) = − [ y · log(ลท) + (1 − y) · log(1 − ลท) ]
Where:
- y is the true label (0 or 1)
- ลท is the predicted probability (between 0 and 1)
BCELoss in PyTorch
In PyTorch, BCELoss is implemented as:
import torch.nn as nn
criterion = nn.BCELoss()
The model output must be passed through a Sigmoid activation
before computing the loss.
BCELoss in GANs
Discriminator
The discriminator is trained to:
- Output 1 for real images
- Output 0 for fake images
loss_real = BCE(D(real_images), 1)
loss_fake = BCE(D(fake_images), 0)
Generator
The generator tries to fool the discriminator by making fake images look real:
loss_generator = BCE(D(fake_images), 1)
Limitations of BCELoss
Although BCELoss is simple and effective, it can cause:
- Vanishing gradients
- Training instability in GANs
Because of this, modern GANs often use alternatives like BCEWithLogitsLoss, Wasserstein loss, or hinge loss.
Summary
Binary Cross-Entropy Loss is a fundamental loss function for binary classification and GAN training. While it is easy to implement and understand, more advanced loss functions are often preferred for improved stability in deep generative models.