Pytorch

Generalities on Pytorch

• torch.function(tensor, args) is equivalent to tensor.function(args).
• Operations can be done inplace (directly modifying the tensor) by adding a _ at the end of the function, e.g., tensor1 = tensor1.add(tensor2) is equivalent to tensor1.add_(tensor2).

Add a dimension to a tensor

With tensor of shape N:

• tensor.unsqueeze(0) has a shape of 1xN
• tensor.unsqueeze(1) has a shape of Nx1

Remove dimension

With tensor of shape 1xN:

• tensor.squeeze(0) has a shape of N

Concatenate tensors

in same dimension :

new_tensor = torch.cat(tensor_list)

in new dimension :

new_tensor = torch.stack(tensor_list)

Tensor to Numpy

A tensor can be transformed to a numpy array using tensor.numpy().

• If the tensor is on a GPU, it must be sent to CPU beforehand with tensor.cpu() (it returns a copy of the tensor on the CPU).
• If the tensor has requires_grad=True, it must be detached from the graph first using tensor.detach() (it returns a copy of the tensor detached from the computing graph).

To transform any tensor to numpy: numpy_array = tensor.cpu().detach().numpy()