Batch
A batch is a group of input examples processed together in one forward and backward pass. Batching improves GPU utilization and stabilizes gradient estimates through averaging.
What is Batch?
A batch is a group of input examples processed together in one forward and backward pass. Batching improves GPU utilization and stabilizes gradient estimates through averaging.
A batch is a group of input examples processed together in one forward and backward pass. Batching improves GPU utilization and stabilizes gradient estimates through averaging.
Where is it used?
LLM pretraining uses batch sizes of millions of tokens (via gradient accumulation); inference batches multiple user requests to maximize throughput on H100 GPUs.
How to build it
Use `DataLoader(dataset, batch_size=32, shuffle=True)`, iterate `for x, y in loader`, and observe GPU memory scaling with batch size via `torch.cuda.max_memory_allocated()`.