Updating the transformers package

Few steps I follow each time i update the transformers package

git pull
pip install --upgrade .
pip install -r ./examples/requirements.txt

that’s

Latex multirow and column

Short and intuitive example

• https://texblog.org/2012/12/21/multi-column-and-multi-row-cells-in-latex-tables/

Numpy RuntimeWarning

Something I learns recently..

• NumPy has its own internal warning architecture on top of Pythons, which can be specifically controlled
• So, something Numpy will just produce a RuntimeWarning  without actually throwing an exception

Consider this:

probs = np.array([0.0, 1.0])
np.prod(probs)**(-1/len(probs))

Numpy produces a RuntimeWarning, not an exception

References:

• https://stackoverflow.com/questions/20161899/scipy-curve-fit-error-divide-by-zero-encountered
• https://stackoverflow.com/questions/10519237/python-how-to-avoid-numpy-runtimewarning-in-function-definition

Gradient accumulation in PyTorch

Need to understand:

• https://medium.com/huggingface/training-larger-batches-practical-tips-on-1-gpu-multi-gpu-distributed-setups-ec88c3e51255
• https://discuss.pytorch.org/t/why-do-we-need-to-set-the-gradients-manually-to-zero-in-pytorch/4903/20
• https://gist.github.com/thomwolf/ac7a7da6b1888c2eeac8ac8b9b05d3d3

 

tiling and repeating tensors

Repeat entire tensor:

• PyTorch:
  • https://pytorch.org/docs/stable/tensors.html#torch.Tensor.repeat

• Numpy:
  • https://numpy.org/doc/stable/reference/generated/numpy.tile.html?highlight=tile

Repeat elements of the tensor

• PyTorch:
  • https://pytorch.org/docs/stable/torch.html#torch.repeat_interleave
• Numpy:
  • https://numpy.org/doc/stable/reference/generated/numpy.repeat.html#numpy.repeat

 

pack_padded_sequence

Nice article on the use of pack_padded_sequence in Pytorch

https://suzyahyah.github.io/pytorch/2019/07/01/DataLoader-Pad-Pack-Sequence.html

 

Ignore tracked files

Often need to do this for appsettings.json files or Tests.runsettings files

Tip:

• https://stackoverflow.com/questions/10755655/git-ignore-tracked-files