Given the memory budget on a GPU, return the largest integer batch size that fits.
Signature: def max_batch_size(weight_bytes: int, activation_bytes_per_sample: int, gpu_vram_bytes: int, overhead_bytes: int) -> int
Return floor((vram - weights - overhead) / activation_per_sample). If the result is negative, return 0.
Math
Asked at
Test Results