2.张量.ipynb

{"cells":[{"cell_type":"markdown","source":["<a href=\"https://github.com/xuehangcang/DeepLearning/blob/main/docs/PyTorch/%E5%9F%BA%E7%A1%80%E7%9F%A5%E8%AF%86/2.张量.ipynb\" download=\"\"><svg xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"110\" height=\"20\" role=\"img\" aria-label=\"jupyter: notebook\"><title>jupyter: notebook</title><linearGradient id=\"s\" x2=\"0\" y2=\"100%\"><stop offset=\"0\" stop-color=\"#bbb\" stop-opacity=\".1\"/><stop offset=\"1\" stop-opacity=\".1\"/></linearGradient><clipPath id=\"r\"><rect width=\"110\" height=\"20\" rx=\"3\" fill=\"#fff\"/></clipPath><g clip-path=\"url(#r)\"><rect width=\"49\" height=\"20\" fill=\"#555\"/><rect x=\"49\" width=\"61\" height=\"20\" fill=\"#fe7d37\"/><rect width=\"110\" height=\"20\" fill=\"url(#s)\"/></g><g fill=\"#fff\" text-anchor=\"middle\" font-family=\"Verdana,Geneva,DejaVu Sans,sans-serif\" text-rendering=\"geometricPrecision\" font-size=\"110\"><text aria-hidden=\"true\" x=\"255\" y=\"150\" fill=\"#010101\" fill-opacity=\".3\" transform=\"scale(.1)\" textLength=\"390\">jupyter</text><text x=\"255\" y=\"140\" transform=\"scale(.1)\" fill=\"#fff\" textLength=\"390\">jupyter</text><text aria-hidden=\"true\" x=\"785\" y=\"150\" fill=\"#010101\" fill-opacity=\".3\" transform=\"scale(.1)\" textLength=\"510\">notebook</text><text x=\"785\" y=\"140\" transform=\"scale(.1)\" fill=\"#fff\" textLength=\"510\">notebook</text></g></svg></a>"],"metadata":{"collapsed":false,"id":"A388871F90E74244AFF0FD31587817B3"}},{"cell_type":"markdown","source":["## 2.1 张量\n","\n","张量是一种特殊的数据结构，非常类似于数组和矩阵。在 PyTorch 中，我们使用张量来编码模型的输入和输出，以及模型的参数。\n","\n","张量类似于 `NumPy` 中的 `ndarrays`，但张量可以在 GPU 或其他硬件加速器上运行。实际上，张量和 NumPy 数组通常可以共享相同的底层内存，消除了复制数据的需要。`张量`还针对自动微分进行了优化。`NumPy` 中的 `ndarrays` 熟悉的话，您会对张量 API 感到亲切。如果不熟悉，请跟随我们的步伐！\n"],"metadata":{"collapsed":false,"id":"822136D43D0445B3BAA2CC50232F63A2"}},{"cell_type":"code","execution_count":8,"outputs":[],"source":["import torch\n","import numpy as np"],"metadata":{"collapsed":false,"ExecuteTime":{"start_time":"2023-04-22T04:09:26.753688Z","end_time":"2023-04-22T04:09:26.768032Z"},"id":"702C0C5E012643CB96CDF77FF90BD583"}},{"cell_type":"markdown","source":["## 2.2 初始化张量\n","\n","张量可以通过多种方式进行初始化。以下是一些例子：\n","\n","**从数据中直接创建**\n","\n","张量可以直接从数据中创建。数据类型会被自动推断。"],"metadata":{"collapsed":false,"id":"DA9AB7D54CCD43A184E84567B5286C33"}},{"cell_type":"code","execution_count":9,"outputs":[{"data":{"text/plain":"tensor([[1, 2],\n        [3, 4]])"},"execution_count":9,"metadata":{},"output_type":"execute_result"}],"source":["data = [[1, 2],[3, 4]]\n","x_data = torch.tensor(data)\n","x_data"],"metadata":{"collapsed":false,"ExecuteTime":{"start_time":"2023-04-22T04:10:43.266041Z","end_time":"2023-04-22T04:10:43.315063Z"},"id":"679EEDDAD6D24FF9A8B15091DE66A85F"}},{"cell_type":"markdown","source":["**从 NumPy 数组创建张量**\n","\n","张量可以从 NumPy 数组创建。"],"metadata":{"collapsed":false,"id":"AE4E525A0F904EC89C486AD058173960"}},{"cell_type":"code","execution_count":10,"outputs":[{"data":{"text/plain":"tensor([[1, 2],\n        [3, 4]], dtype=torch.int32)"},"execution_count":10,"metadata":{},"output_type":"execute_result"}],"source":["np_array = np.array(data)\n","x_np = torch.from_numpy(np_array)\n","x_np"],"metadata":{"collapsed":false,"ExecuteTime":{"start_time":"2023-04-22T04:11:28.403721Z","end_time":"2023-04-22T04:11:28.419752Z"},"id":"ACD5BA2CF9A745ECBC4F716C272F155B"}},{"cell_type":"markdown","source":["**来自另一个张量**\n","\n","新的张量保留了参数张量的属性（形状、数据类型），除非明确覆盖。"],"metadata":{"collapsed":false,"id":"83692B678E364A6C96A5261631FF6535"}},{"cell_type":"code","execution_count":11,"outputs":[{"name":"stdout","output_type":"stream","text":["Ones Tensor: \n"," tensor([[1, 1],\n","        [1, 1]]) \n","\n","Random Tensor: \n"," tensor([[0.5579, 0.5803],\n","        [0.3633, 0.7775]]) \n","\n"]}],"source":["x_ones = torch.ones_like(x_data) # 保留了x_data的属性\n","print(f\"Ones Tensor: \\n {x_ones} \\n\")\n","\n","x_rand = torch.rand_like(x_data, dtype=torch.float) # 覆盖了x_data的数据类型\n","print(f\"Random Tensor: \\n {x_rand} \\n\")"],"metadata":{"collapsed":false,"ExecuteTime":{"start_time":"2023-04-22T04:12:21.960335Z","end_time":"2023-04-22T04:12:22.006472Z"},"id":"42BD34D537164588B1C06DA3FD0F1C9E"}},{"cell_type":"markdown","source":["**随机或常数值：**\n","\n","`shape` 是张量维度的元组。在下面的函数中，它决定了输出张量的维度。"],"metadata":{"collapsed":false,"id":"4E3DDA6C3B07414BB44A3C8B8BF61EE3"}},{"cell_type":"code","execution_count":12,"outputs":[{"name":"stdout","output_type":"stream","text":["Random Tensor: \n"," tensor([[0.1944, 0.9346, 0.8559],\n","        [0.7070, 0.1307, 0.4355]]) \n","\n","Ones Tensor: \n"," tensor([[1., 1., 1.],\n","        [1., 1., 1.]]) \n","\n","Zeros Tensor: \n"," tensor([[0., 0., 0.],\n","        [0., 0., 0.]])\n"]}],"source":["shape = (2,3,)\n","rand_tensor = torch.rand(shape)\n","ones_tensor = torch.ones(shape)\n","zeros_tensor = torch.zeros(shape)\n","\n","print(f\"Random Tensor: \\n {rand_tensor} \\n\")\n","print(f\"Ones Tensor: \\n {ones_tensor} \\n\")\n","print(f\"Zeros Tensor: \\n {zeros_tensor}\")"],"metadata":{"collapsed":false,"ExecuteTime":{"start_time":"2023-04-22T04:12:53.825664Z","end_time":"2023-04-22T04:12:53.870395Z"},"id":"01446591397C48C5BFCAF74339E62A86"}},{"cell_type":"markdown","source":["## 2.3 张量的属性\n","\n","张量的属性描述了它们的形状、数据类型和存储设备。\n"],"metadata":{"collapsed":false,"id":"B4A6C8BD9A3A430C95A700A752FACC46"}},{"cell_type":"code","execution_count":13,"outputs":[{"name":"stdout","output_type":"stream","text":["Shape of tensor: torch.Size([3, 4])\n","Datatype of tensor: torch.float32\n","Device tensor is stored on: cpu\n"]}],"source":["tensor = torch.rand(3,4)\n","\n","print(f\"Shape of tensor: {tensor.shape}\")\n","print(f\"Datatype of tensor: {tensor.dtype}\")\n","print(f\"Device tensor is stored on: {tensor.device}\")"],"metadata":{"collapsed":false,"ExecuteTime":{"start_time":"2023-04-22T04:13:28.298119Z","end_time":"2023-04-22T04:13:28.314339Z"},"id":"9C7258194D3740A5B692E2DFC738D8B6"}},{"cell_type":"markdown","source":["## 2.4 张量的操作\n","\n","PyTorch 提供了 100 多种张量操作，包括算术、线性代数、矩阵操作（转置、索引、切片）、采样等等，全部在 `https://pytorch.org/docs/stable/torch.html>` 进行了全面的描述。\n","\n","每个操作都可以在GPU上运行（通常比在CPU上运行速度更快）。\n","\n","默认情况下，张量在CPU上创建。我们需要使用 `.to` 方法将张量显式地移动到 GPU 上（在检查GPU可用性之后）。请记住，跨设备复制大型张量可能会花费大量的时间和内存！\n","\n","如果可用，我们将张量移动到GPU上"],"metadata":{"collapsed":false,"id":"9B6FC593A8A443B3B82BD6047FD4D249"}},{"cell_type":"code","execution_count":14,"outputs":[],"source":["if torch.cuda.is_available():\n","    tensor = tensor.to(\"cuda\")"],"metadata":{"collapsed":false,"ExecuteTime":{"start_time":"2023-04-22T04:15:01.578102Z","end_time":"2023-04-22T04:15:01.728067Z"},"id":"C17EDEC35912491492457B0CDB32349F"}},{"cell_type":"markdown","source":["尝试使用列表中的一些操作。\n","如果您熟悉NumPy API，您会发现Tensor API非常易于使用。\n","\n"," **标准的类似于NumPy的索引和切片操作**"],"metadata":{"collapsed":false,"id":"888E853BE946450B998FFA6F8D678D56"}},{"cell_type":"code","execution_count":15,"outputs":[{"name":"stdout","output_type":"stream","text":["First row: tensor([1., 1., 1., 1.])\n","First column: tensor([1., 1., 1., 1.])\n","Last column: tensor([1., 1., 1., 1.])\n","tensor([[1., 0., 1., 1.],\n","        [1., 0., 1., 1.],\n","        [1., 0., 1., 1.],\n","        [1., 0., 1., 1.]])\n"]}],"source":["tensor = torch.ones(4, 4)\n","print(f\"First row: {tensor[0]}\")\n","print(f\"First column: {tensor[:, 0]}\")\n","print(f\"Last column: {tensor[..., -1]}\")\n","tensor[:,1] = 0\n","print(tensor)"],"metadata":{"collapsed":false,"ExecuteTime":{"start_time":"2023-04-22T04:15:41.870374Z","end_time":"2023-04-22T04:15:41.892996Z"},"id":"6D2F977D74404BA9A8B44D6F5CC722D7"}},{"cell_type":"markdown","source":["**拼接张量**\n","\n","您可以使用 `torch.cat` 沿着给定维度连接一系列张量。"],"metadata":{"collapsed":false,"id":"26E2925ACE5F4F389E1DCF13EEA9F26D"}},{"cell_type":"code","execution_count":16,"outputs":[{"name":"stdout","output_type":"stream","text":["tensor([[1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.],\n","        [1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.],\n","        [1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.],\n","        [1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.]])\n"]}],"source":["t1 = torch.cat([tensor, tensor, tensor], dim=1)\n","print(t1)"],"metadata":{"collapsed":false,"ExecuteTime":{"start_time":"2023-04-22T04:16:24.249608Z","end_time":"2023-04-22T04:16:24.266453Z"},"id":"B94F4DB47F40461288A9C6C2DC6EF736"}},{"cell_type":"markdown","source":["**算术运算**\n","\n","这将计算两个张量之间的矩阵乘法。y1、y2和y3将具有相同的值。\n","`tensor.T` 返回张量的转置。"],"metadata":{"collapsed":false,"id":"EAD51B279496484C93BA1CA146FE7C29"}},{"cell_type":"code","execution_count":17,"outputs":[{"data":{"text/plain":"tensor([[3., 3., 3., 3.],\n        [3., 3., 3., 3.],\n        [3., 3., 3., 3.],\n        [3., 3., 3., 3.]])"},"execution_count":17,"metadata":{},"output_type":"execute_result"}],"source":["y1 = tensor @ tensor.T\n","y2 = tensor.matmul(tensor.T)\n","\n","y3 = torch.rand_like(y1)\n","torch.matmul(tensor, tensor.T, out=y3)"],"metadata":{"collapsed":false,"ExecuteTime":{"start_time":"2023-04-22T04:17:02.537085Z","end_time":"2023-04-22T04:17:02.584772Z"},"id":"DF8BEF2C02FB4114B154770B65DFFA5B"}},{"cell_type":"markdown","source":["这将计算逐元素乘积。z1、z2、z3 将具有相同的值。"],"metadata":{"collapsed":false,"id":"5CA7C9D845BC4F48A978EA83AF8064A2"}},{"cell_type":"code","execution_count":18,"outputs":[{"data":{"text/plain":"tensor([[1., 0., 1., 1.],\n        [1., 0., 1., 1.],\n        [1., 0., 1., 1.],\n        [1., 0., 1., 1.]])"},"execution_count":18,"metadata":{},"output_type":"execute_result"}],"source":["z1 = tensor * tensor\n","z2 = tensor.mul(tensor)\n","\n","z3 = torch.rand_like(tensor)\n","torch.mul(tensor, tensor, out=z3)\n"],"metadata":{"collapsed":false,"ExecuteTime":{"start_time":"2023-04-22T04:17:23.374282Z","end_time":"2023-04-22T04:17:23.421742Z"},"id":"A29772E32E6C4F08A9102CA681EDDFEF"}},{"cell_type":"markdown","source":["**单元素张量**\n","\n","如果你有一个只有一个元素的张量，例如将张量中的所有值聚合为一个值，你可以使用 `item()` 将其转换为 `Python` 数值。"],"metadata":{"collapsed":false,"id":"EA39E8BFCD79499191063FA313E1AFA7"}},{"cell_type":"code","execution_count":19,"outputs":[{"name":"stdout","output_type":"stream","text":["12.0 <class 'float'>\n"]}],"source":["agg = tensor.sum()\n","agg_item = agg.item()\n","print(agg_item, type(agg_item))"],"metadata":{"collapsed":false,"ExecuteTime":{"start_time":"2023-04-22T04:18:06.077537Z","end_time":"2023-04-22T04:18:06.095724Z"},"id":"039236149F2A42EB869219DBAE41C4B4"}},{"cell_type":"markdown","source":["**原地操作**\n","\n","将结果存储到操作数中的操作称为原地操作。它们用 `_` 后缀表示。 虽然就地操作可以节省一些内存，但是在计算导数时可能会出现问题，因为会立即丢失历史记录。因此，我们不建议使用就地操作。\n","\n","例如：`x.copy_(y)`, `x.t_()` 将会改变 `x`。"],"metadata":{"collapsed":false,"id":"A91A208CB9B74ECDA65C88E261ED2D9F"}},{"cell_type":"code","execution_count":21,"outputs":[{"name":"stdout","output_type":"stream","text":["tensor([[6., 5., 6., 6.],\n","        [6., 5., 6., 6.],\n","        [6., 5., 6., 6.],\n","        [6., 5., 6., 6.]]) \n","\n","tensor([[11., 10., 11., 11.],\n","        [11., 10., 11., 11.],\n","        [11., 10., 11., 11.],\n","        [11., 10., 11., 11.]])\n"]}],"source":["print(f\"{tensor} \\n\")\n","tensor.add_(5)\n","print(tensor)"],"metadata":{"collapsed":false,"ExecuteTime":{"start_time":"2023-04-22T04:19:32.228042Z","end_time":"2023-04-22T04:19:32.247162Z"},"id":"136DB5C563AD41FA9CB6CFF9C3C944CF"}},{"cell_type":"markdown","source":["## 2.5 使用 NumPy 进行桥接\n","\n"," CPU 上的张量和 NumPy 数组可以共享它们的底层内存位置，改变其中一个将会改变另一个。"],"metadata":{"collapsed":false,"id":"6C8ADCA6AA6B439DA2A36262239E1A84"}},{"cell_type":"markdown","source":["### 2.5.1 将Tensor转换为NumPy数组"],"metadata":{"collapsed":false,"id":"164FB3F6A56C430C8040841862488EC9"}},{"cell_type":"code","execution_count":22,"outputs":[{"name":"stdout","output_type":"stream","text":["t: tensor([1., 1., 1., 1., 1.])\n","n: [1. 1. 1. 1. 1.]\n"]}],"source":["t = torch.ones(5)\n","print(f\"t: {t}\")\n","n = t.numpy()\n","print(f\"n: {n}\")"],"metadata":{"collapsed":false,"ExecuteTime":{"start_time":"2023-04-22T04:20:25.690559Z","end_time":"2023-04-22T04:20:25.709089Z"},"id":"CC0FFB938C924AB0A9BBDA690B81833C"}},{"cell_type":"markdown","source":["张量的变化会反映在 NumPy 数组中。"],"metadata":{"collapsed":false,"id":"180AA21E73BF40919BAE9F4A01044830"}},{"cell_type":"code","execution_count":23,"outputs":[{"name":"stdout","output_type":"stream","text":["t: tensor([2., 2., 2., 2., 2.])\n","n: [2. 2. 2. 2. 2.]\n"]}],"source":["t.add_(1)\n","print(f\"t: {t}\")\n","print(f\"n: {n}\")"],"metadata":{"collapsed":false,"ExecuteTime":{"start_time":"2023-04-22T04:20:49.195322Z","end_time":"2023-04-22T04:20:49.239945Z"},"id":"4CC469DAD22E44159DAFBA815073D062"}},{"cell_type":"markdown","source":["### 2.5.2 将NumPy数组转换为张量"],"metadata":{"collapsed":false,"id":"9AFD97C37E3D4E438BB3D67619C989E9"}},{"cell_type":"code","execution_count":25,"outputs":[{"data":{"text/plain":"tensor([1., 1., 1., 1., 1.], dtype=torch.float64)"},"execution_count":25,"metadata":{},"output_type":"execute_result"}],"source":["n = np.ones(5)\n","t = torch.from_numpy(n)\n","t"],"metadata":{"collapsed":false,"ExecuteTime":{"start_time":"2023-04-22T04:21:51.288799Z","end_time":"2023-04-22T04:21:51.337344Z"},"id":"F6A6A60DA9564956AC837E662F2C1328"}},{"cell_type":"markdown","source":["NumPy数组的变化会反映在张量中。"],"metadata":{"collapsed":false,"id":"9D3D1512C27E4C89A86AAEF7A33874E6"}},{"cell_type":"code","execution_count":26,"outputs":[{"name":"stdout","output_type":"stream","text":["t: tensor([2., 2., 2., 2., 2.], dtype=torch.float64)\n","n: [2. 2. 2. 2. 2.]\n"]}],"source":["np.add(n, 1, out=n)\n","print(f\"t: {t}\")\n","print(f\"n: {n}\")"],"metadata":{"collapsed":false,"ExecuteTime":{"start_time":"2023-04-22T04:22:12.666144Z","end_time":"2023-04-22T04:22:12.684971Z"},"id":"4849159339ED4F1C9883E58466536E1F"}}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python","nbconvert_exporter":"python","file_extension":".py","version":"3.5.2","pygments_lexer":"ipython3"}},"nbformat":4,"nbformat_minor":0}