Beyond Integer indexing

Faced an interesting problem recently

a : (B, S, T)
b : (B, C)  where 0 <= x[i, j] < S

What I want is an array of shape (B, C, T)

a = np.array(
   ...:    [[[0,1,2,3], 
   ...:      [4,5,6,7],
   ...:      [8,9,10,11]],
   ...:     [[0,1,2,3],
   ...:      [4,5,6,7],
   ...:      [8,9,10,11]]])

b = np.array(
   ...:    [[0,2,2],
   ...:     [1,0, 2]])

a.shape
Out[79]: (2, 3, 4)

b.shape
Out[80]: (2, 3)

What I expect is this

array([[[ 0,  1,  2,  3],
        [ 8,  9, 10, 11],
        [ 8,  9, 10, 11]],
       [[ 4,  5,  6,  7],
        [ 0,  1,  2,  3],
        [ 8,  9, 10, 11]]])

Note this is different from the typical scenario

Initially I hit some issues with integer index broadcasting. It seems it is possible to do it.

a[np.array([np.arange(2)]).T, b]

References:

• https://stackoverflow.com/questions/46124469/shape-mismatch-indexing-arrays-could-not-be-broadcast-together-with-shapes
• https://docs.scipy.org/doc/numpy-1.13.0/user/basics.indexing.html#indexing-multi-dimensional-arrays

tiling and repeating tensors

Repeat entire tensor:

• PyTorch:
  • https://pytorch.org/docs/stable/tensors.html#torch.Tensor.repeat

• Numpy:
  • https://numpy.org/doc/stable/reference/generated/numpy.tile.html?highlight=tile

Repeat elements of the tensor

• PyTorch:
  • https://pytorch.org/docs/stable/torch.html#torch.repeat_interleave
• Numpy:
  • https://numpy.org/doc/stable/reference/generated/numpy.repeat.html#numpy.repeat

 

Frequency Counting in Python.

One of the most frequent operations when doing data analysis is looking at the frequency counts information.

I wanted to list down the various ways of doing this task:

• using python collections: Counter and Defaultdict
• using numpy
  • with numpy.unique, with return_counts argument
  • with bincount, nonzero, zip / vstack
• using pandas
• using scipy

 

References:

• http://stackoverflow.com/questions/2870466/python-histogram-one-liner
• http://stackoverflow.com/questions/10741346/numpy-most-efficient-frequency-counts-for-unique-values-in-an-array

Code:

• freqCounts

 

 

Getting started with Numpy

One of the best getting started guides on Numpy is the stanford tutorial.

http://cs231n.github.io/python-numpy-tutorial/

For numpy broadcasting, this is a great guide:

• http://scipy.github.io/old-wiki/pages/EricsBroadcastingDoc
• Very helpful!

 

References:

• http://www.python-course.eu/numpy.php
• http://docs.scipy.org/doc/numpy/user/basics.creation.html
• http://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.fill.html
• http://docs.scipy.org/doc/numpy/reference/generated/numpy.full.html