Ben Chuanlong Du's Blog

It is never too late to learn.

Resize a Tensor in PyTorch

In [2]:
import torch
In [13]:
x = torch.tensor(
    [
        [1, 2, 3, 4, 5],
        [6, 7, 8, 9, 10],
    ]
)
x
Out[13]:
tensor([[ 1,  2,  3,  4,  5],
        [ 6,  7,  8,  9, 10]])
In [44]:
x.shape
Out[44]:
torch.Size([2, 5])

Tensor.resize_

Notice that Tensor.resize_ is in-place and the non in-place version Tensor.resize is deprecated.

In [45]:
xc = x.clone()
xc
Out[45]:
tensor([[ 1,  2,  3,  4,  5],
        [ 6,  7,  8,  9, 10]])
In [46]:
xc is x
Out[46]:
False
In [47]:
xc.resize_((1, 2, 5))
Out[47]:
tensor([[[ 1,  2,  3,  4,  5],
         [ 6,  7,  8,  9, 10]]])
In [48]:
xc.shape
Out[48]:
torch.Size([1, 2, 5])

You can resize a Tensor to have more elements. The Tensor is padded on the right.

  • A float Tensor is padded with the value 0.
  • It seems that an int Tensor is padded with random integers.

Resize an int Tensor.

In [8]:
t = torch.tensor([1, 2, 3])
t
Out[8]:
tensor([1, 2, 3])
In [9]:
t.resize_(5)
Out[9]:
tensor([  1,   2,   3, 112,   0])

Resize a float Tensor.

In [10]:
t = torch.tensor([1.0, 2.0, 3.0])
t
Out[10]:
tensor([1., 2., 3.])
In [11]:
t.resize_(5)
Out[11]:
tensor([1.0000e+00, 2.0000e+00, 3.0000e+00, 4.5862e-41, 8.9683e-44])

Tensor.squeeze

In [49]:
x.squeeze(0)
Out[49]:
tensor([[ 1,  2,  3,  4,  5],
        [ 6,  7,  8,  9, 10]])
In [50]:
x.squeeze(0).shape
Out[50]:
torch.Size([2, 5])
In [51]:
y = x.unsqueeze(0)
y
Out[51]:
tensor([[[ 1,  2,  3,  4,  5],
         [ 6,  7,  8,  9, 10]]])
In [52]:
y.shape
Out[52]:
torch.Size([1, 2, 5])
In [53]:
y.squeeze(0)
Out[53]:
tensor([[ 1,  2,  3,  4,  5],
        [ 6,  7,  8,  9, 10]])
In [54]:
y.squeeze(0).shape
Out[54]:
torch.Size([2, 5])

Tensor.unsqueeze

In [55]:
x.unsqueeze(0)
Out[55]:
tensor([[[ 1,  2,  3,  4,  5],
         [ 6,  7,  8,  9, 10]]])
In [56]:
x.unsqueeze(0).shape
Out[56]:
torch.Size([1, 2, 5])

Tensor.view

In [58]:
x.view((1, 2, 5))
Out[58]:
tensor([[[ 1,  2,  3,  4,  5],
         [ 6,  7,  8,  9, 10]]])
In [59]:
x.view((1, 2, 5)).shape
Out[59]:
torch.Size([1, 2, 5])

Tensor.view_as

View this tensor as the same size as other. self.view_as(other) is equivalent to self.view(other.size()).

Tensor.expand

Returns a new view of the self tensor with singleton dimensions expanded to a larger size.

Tensor.expand is used to replicate data in a tensor. If x is the tensor to be expanded. The new shape must be (k, *x.shape), where k is a non-negative integer.

In [14]:
x
Out[14]:
tensor([[ 1,  2,  3,  4,  5],
        [ 6,  7,  8,  9, 10]])
In [60]:
x.expand((3, 2, 5))
Out[60]:
tensor([[[ 1,  2,  3,  4,  5],
         [ 6,  7,  8,  9, 10]],

        [[ 1,  2,  3,  4,  5],
         [ 6,  7,  8,  9, 10]],

        [[ 1,  2,  3,  4,  5],
         [ 6,  7,  8,  9, 10]]])
In [61]:
x.expand((1, 2, 5))
Out[61]:
tensor([[[ 1,  2,  3,  4,  5],
         [ 6,  7,  8,  9, 10]]])
In [62]:
x.expand((1, 2, 5)).shape
Out[62]:
torch.Size([1, 2, 5])

Tensor.repeat

In [21]:
x = torch.tensor([1, 2, 3])
x
Out[21]:
tensor([1, 2, 3])
In [23]:
x.repeat(2)
Out[23]:
tensor([1, 2, 3, 1, 2, 3])
In [25]:
x.repeat(2, 4, 3)
Out[25]:
tensor([[[1, 2, 3, 1, 2, 3, 1, 2, 3],
         [1, 2, 3, 1, 2, 3, 1, 2, 3],
         [1, 2, 3, 1, 2, 3, 1, 2, 3],
         [1, 2, 3, 1, 2, 3, 1, 2, 3]],

        [[1, 2, 3, 1, 2, 3, 1, 2, 3],
         [1, 2, 3, 1, 2, 3, 1, 2, 3],
         [1, 2, 3, 1, 2, 3, 1, 2, 3],
         [1, 2, 3, 1, 2, 3, 1, 2, 3]]])

Add a Dummy Dimension

If you need to add a dummy dimension at the beginning or the end, it is the easiest to use the method Tensor.unsqueeze since you only need to specify the position to add the dummy dimension instead of specifying the full new dimension. Otherwise, I'd suggest you to use the method Tensor.view as it can be used for both adding/removing a dummy dimension (at the slight inconvenience of specifying the full new dimension).

In [63]:
x.expand((1, 2, 5))
Out[63]:
tensor([[[ 1,  2,  3,  4,  5],
         [ 6,  7,  8,  9, 10]]])
In [64]:
x.view((1, 2, 5))
Out[64]:
tensor([[[ 1,  2,  3,  4,  5],
         [ 6,  7,  8,  9, 10]]])
In [65]:
x.unsqueeze(0)
Out[65]:
tensor([[[ 1,  2,  3,  4,  5],
         [ 6,  7,  8,  9, 10]]])
In [66]:
x.clone().resize_((1, 2, 5))
Out[66]:
tensor([[[ 1,  2,  3,  4,  5],
         [ 6,  7,  8,  9, 10]]])

You can also use numpy-style slicing to add a dummy dimenson!!!

In [75]:
x[None, :, :]
Out[75]:
tensor([[[ 1,  2,  3,  4,  5],
         [ 6,  7,  8,  9, 10]]])

Remove a Dummy Dimension

If you need to remove a dummy dimension from the beginning or the end, it is the easiest to use the method Tensor.squeeze since you only need to specify the position to remove the dummy dimension instead of specifying the full new dimension. Otherwise, I'd suggest you to use the method Tensor.view as it can be used for both adding/removing a dummy dimension (at the slight inconvenience of specifying the full new dimension).

In [67]:
y = x.unsqueeze(0)
y
Out[67]:
tensor([[[ 1,  2,  3,  4,  5],
         [ 6,  7,  8,  9, 10]]])
In [68]:
y.shape
Out[68]:
torch.Size([1, 2, 5])
In [70]:
y.view((2, 5))
Out[70]:
tensor([[ 1,  2,  3,  4,  5],
        [ 6,  7,  8,  9, 10]])
In [71]:
y.squeeze(0)
Out[71]:
tensor([[ 1,  2,  3,  4,  5],
        [ 6,  7,  8,  9, 10]])
In [72]:
y.clone().resize_((2, 5))
Out[72]:
tensor([[ 1,  2,  3,  4,  5],
        [ 6,  7,  8,  9, 10]])

References

https://discuss.pytorch.org/t/what-is-the-difference-between-view-and-unsqueeze/1155

https://mlpipes.com/adding-a-dimension-to-a-tensor-in-pytorch/

https://stackoverflow.com/questions/43328632/pytorch-reshape-tensor-dimension

Comments