Hands on Tensor in PyTorch

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

import torch
import numpy as np

x = torch.tensor(
    [
        [1.0, 2, 3, 4, 5],
        [6.0, 7, 8, 9, 10],
    ]
)
x

tensor([[ 1.,  2.,  3.,  4.,  5.],
        [ 6.,  7.,  8.,  9., 10.]])

torch.Tensor vs torch.tensor vs torch.as_tensor¶

torch.Tensor always returns torch.FloatTensor. torch.tensor infers the data type and allows users to specify the data type. It is suggested that you use torch.tensor instead of torch.Tensor.
torch.tensor always copies data while torch.as_tensor avoids copying data if possible. One such an example is when you convert a numpy array to a Tensor. However, notice that both torch.tensor and torch.as_tensor copies data if a list is feeded to them.
In most situations, you should use torch.tensor. Never use torch.Tensor. Be cautious if you use torch.as_tensor.

a1 = np.array([1, 2, 3])
t1 = torch.Tensor(arr)
t1

tensor([1., 2., 3.])

a1[0] = 1000
t1

tensor([1., 2., 3.])

a2 = np.array([1, 2, 3])
t2 = torch.Tensor(arr)
t2

tensor([1., 2., 3.])

a2[0] = 1000
t2

tensor([1., 2., 3.])

a3 = np.array([1, 2, 3])
t3 = torch.as_tensor(a3)
t3

tensor([1, 2, 3])

a3[0] = 1000
t3

tensor([1000, 2, 3])

torch.rand¶

torch.rand(10)

tensor([0.7872, 0.1416, 0.2451, 0.7463, 0.3549, 0.6301, 0.3934, 0.4746, 0.0036,
        0.0931])

torch.stack¶

Concatenates sequence of tensors along a new dimension. All tensors need to be of the same size.

data = [torch.tensor([1, 2, 3]), torch.tensor([4, 5, 6])]

torch.stack(data)

tensor([[1, 2, 3],
        [4, 5, 6]])

torch.stack takes a list/tuple of tensors. It does NOT work on a generator/iterator of tensors.

torch.stack(t for t in data)

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-6-4a0a55595f8d> in <module>
----> 1 torch.stack(t for t in data)

TypeError: stack(): argument 'tensors' (position 1) must be tuple of Tensors, not generator

Tensor.detach¶

Returns a new Tensor, detached from the current graph.

The result will never require gradient.

Note: Returned Tensor shares the same storage with the original one. In-place modifications on either of them will be seen, and may trigger errors in correctness checks. IMPORTANT NOTE: Previously, in-place size / stride / storage changes (such as resize_ / resize_as_ / set_ / transpose_) to the returned tensor also update the original tensor. Now, these in-place changes will not update the original tensor anymore, and will instead trigger an error. For sparse tensors: In-place indices / values changes (such as zero_ / copy_ / add_) to the returned tensor will not update the original tensor anymore, and will instead trigger an error.

?x.detach

Docstring:
Returns a new Tensor, detached from the current graph.

The result will never require gradient.

.. note::

  Returned Tensor shares the same storage with the original one.
  In-place modifications on either of them will be seen, and may trigger
  errors in correctness checks.
  IMPORTANT NOTE: Previously, in-place size / stride / storage changes
  (such as `resize_` / `resize_as_` / `set_` / `transpose_`) to the returned tensor
  also update the original tensor. Now, these in-place changes will not update the
  original tensor anymore, and will instead trigger an error.
  For sparse tensors:
  In-place indices / values changes (such as `zero_` / `copy_` / `add_`) to the
  returned tensor will not update the original tensor anymore, and will instead
  trigger an error.
Type:      builtin_function_or_method

Tensor.mean¶

x.mean()

tensor(5.5000)

Tensor.item¶

x.item()

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-7-3396a1b2b617> in <module>
----> 1 x.item()

ValueError: only one element tensors can be converted to Python scalars

y = torch.tensor([2])
y

tensor([2])

y.item()

2

y = torch.tensor(2)
y

tensor(2)

y.item()

2

Rerferences¶

torch.Tensor

torch.as_tensor

torch.tensor

torch.as_tensor fails to create named tensors

torch.Tensor vs torch.tensor vs torch.as_tensor¶

torch.rand¶

torch.stack¶

Tensor.detach¶

Tensor.mean¶

Tensor.item¶

Tensor.backward¶

Move a Tensor to a Specific Device in PyTorch¶

Convert a Tensor to a Numpy Array in PyTorch¶

Tensor Transformations in TorchVision¶

Resize a Tensor in PyTorch¶

Rerferences¶