Gradient accumulation in PyTorch

Need to understand:

 

tiling and repeating tensors

Repeat entire tensor:

Repeat elements of the tensor