diff --git a/MindFlow/applications/research/NSFNets/NSFNets.ipynb b/MindFlow/applications/research/NSFNets/NSFNets.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..38a01b126619dc859dc565f142ff1ac128d7896a --- /dev/null +++ b/MindFlow/applications/research/NSFNets/NSFNets.ipynb @@ -0,0 +1,414 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "de7a09e7-32fc-4059-9478-6c6fc6667f57", + "metadata": {}, + "source": [ + "# NSFNets: Physics-informed neural networks for the incompressible Navier-Stokes equations\n", + "\n", + "## Environment Setup\n", + "\n", + "This notebook requires **MindSpore version >= 2.0.0** to support new APIs including: *mindspore.jit, mindspore.jit_class, mindspore.data_sink*. Please check [MindSpore Installation](https://www.mindspore.cn/install/en) for details.\n", + "\n", + "In addition, **MindFlow version >=0.1.0** is also required. If it has not been installed in your environment, please select the right version and hardware, then install it as follows." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "5a449343-731b-46bb-b8dd-f7a8c1ba8071", + "metadata": {}, + "outputs": [], + "source": [ + "mindflow_version = \"0.1.0\" # update if needed\n", + "# GPU Comment out the following code if you are using NPU.\n", + "!pip uninstall -y mindflow-gpu\n", + "!pip install mindflow-gpu==$mindflow_version\n", + "\n", + "# NPU Uncomment if needed.\n", + "# !pip uninstall -y mindflow-ascend\n", + "# !pip install mindflow-ascend==$mindflow_version" + ] + }, + { + "cell_type": "markdown", + "id": "25fe4081-9d91-44c1-b7de-b50a7645e090", + "metadata": {}, + "source": [ + "## Background\n", + "\n", + "In the industrial application of AI-CFD, the scenarios for solving the NavierStokes equations and fusing multi-precision data are very extensive and have important economic and social significance. Solving ill-posed problems (such as missing some boundary conditions) or inversion problems is one of the key points and difficulties, and it is often costly, requiring the derivation of different formulas applicable to specific problems and the writing of new codes. How to solve the above problems with a set of unified codes at the same computational cost urgently needs in-depth research. Here, Jin Xiaowei, Li Hui and others used physical information neural networks (PINNs) to encode the control equations directly into deep neural networks through automatic differentiation to overcome some of the above-mentioned limitations of simulating incompressible laminar and turbulent flows. And developed Navier-Stokes flow networks (NSFnets, Navier-Stokes flow nets)." + ] + }, + { + "cell_type": "markdown", + "id": "50540e43-3c8e-45f0-8e2b-78831ec89684", + "metadata": {}, + "source": [ + "## Model framework\n", + "\n", + "The model framework is as shown in the following figure:\n", + "\n", + "![NSFNet](images/NSFNet.png)\n", + "\n", + "The figure shows the network for solving the Navier-Stokes equations in the pressure-velocity form. The automatic differentiation of the neural network is used to calculate the partial derivatives required in the equation. The loss function includes boundary condition loss, initial condition loss, and physical loss to satisfy the balance of the equation.\n", + "\n", + "## Preparation\n", + "\n", + "Before practice, ensure that MindSpore of suitable version has been correctly installed. If not, you can run the following command:\n", + "\n", + "* [MindSpore installation page](https://www.mindspore.cn/install) Install MindSpore.\n", + "\n", + "## Datasets Preparation\n", + "\n", + "The dataset can be directly generated by the provided code." + ] + }, + { + "cell_type": "markdown", + "id": "23990483-d613-4c8b-aa7e-0a8b16a89b13", + "metadata": {}, + "source": [ + "## Model Training\n", + "\n", + "Import code packs." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "e3d17b8a-bde1-4ccd-9345-ccfaa2cd1bfe", + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "import argparse\n", + "\n", + "import mindspore as ms\n", + "import time\n", + "import numpy as np\n", + "from mindspore import set_seed, context, nn, Tensor\n", + "from src.network import VPNSFNets\n", + "from src.datasets import read_training_data" + ] + }, + { + "cell_type": "markdown", + "id": "0b8e1d64-580b-4ae9-a7f6-164b0ee0adf3", + "metadata": {}, + "source": [ + "Setting of model-related parameters and definition of training model" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "8a84c0ff-d4cf-41dd-8afc-db8f234d2108", + "metadata": {}, + "outputs": [], + "source": [ + "parser = argparse.ArgumentParser()\n", + "parser.add_argument('--model_name', type=str, default='NSFNet')\n", + "parser.add_argument('--case', type=str, default='Three dimensional Beltrami flow')\n", + "parser.add_argument('--device', type=str, default='GPU')\n", + "parser.add_argument('--device_id', type=str, default='1')\n", + "parser.add_argument('--load_params', type=str, default=False)\n", + "parser.add_argument('--second_path', type=str, default='train')\n", + "parser.add_argument('--network_size', type=int, default=[4] + 10 * [100 * 1] + [4])\n", + "parser.add_argument('--learning_rate', type=int, default=[1.0e-03, 1.0e-04, 1.0e-05, 1.0e-06])\n", + "parser.add_argument('--epochs', type=int, default=[5e3, 5e3, 5e4, 5e4])\n", + "parser.add_argument('--batch_size', type=int, default=10000)\n", + "parser.add_argument('--re', type=int, default=1)\n", + "args = parser.parse_known_args()[0]\n", + "# args = parser.parse_args()\n", + "\n", + "model_name = args.model_name\n", + "case = args.case\n", + "device = args.device\n", + "device_id = args.device_id\n", + "network_size = args.network_size\n", + "learning_rate = args.learning_rate\n", + "epochs = args.epochs\n", + "batch_size = args.batch_size\n", + "load_params = args.load_params\n", + "second_path = args.second_path\n", + "re = args.re\n", + "\n", + "use_ascend = context.get_context(attr_key='device_target') == \"Ascend\"\n", + "\n", + "if use_ascend:\n", + " msfloat_type = ms.float16\n", + " npfloat_type = np.float16\n", + "else:\n", + " msfloat_type = ms.float32\n", + " npfloat_type = np.float32\n", + "\n", + "os.environ['CUDA_VISIBLE_DEVICES'] = device_id\n", + "context.set_context(mode=context.GRAPH_MODE, save_graphs=False, device_target=device)\n", + "\n", + "x_b, y_b, z_b, t_b, x_i, y_i, z_i, t_i, u_b, v_b, w_b, u_i, v_i, w_i, x_f, y_f, z_f, t_f, X_min, X_max = read_training_data()\n", + "\n", + "model = VPNSFNets(x_b, y_b, z_b, t_b, x_i, y_i, z_i, t_i, u_b, v_b, w_b, u_i, v_i, w_i, x_f, y_f, z_f, t_f, network_size, re, \\\n", + " X_min, X_max, use_ascend, msfloat_type, npfloat_type, load_params, second_path)" + ] + }, + { + "cell_type": "markdown", + "id": "387cb210-078e-41e1-b960-e432af9c8d5f", + "metadata": {}, + "source": [ + "Record the changes in loss." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "482e625a-262a-4f06-a4a7-834766ed6ab4", + "metadata": {}, + "outputs": [], + "source": [ + "# Adam loss history\n", + "loss_history_Adam_Pretrain = np.empty([0])\n", + "loss_b_history_Adam_Pretrain = np.empty([0])\n", + "loss_i_history_Adam_Pretrain = np.empty([0])\n", + "loss_f_history_Adam_Pretrain = np.empty([0])\n", + "np.random.seed(123456)\n", + "set_seed(123456)" + ] + }, + { + "cell_type": "markdown", + "id": "30f8c191-ea21-4055-9d32-dfae30c96a33", + "metadata": {}, + "source": [ + "Code training and output results." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "cff1733f-6e6a-47ff-928f-be3a348e98da", + "metadata": {}, + "outputs": [], + "source": [ + "def train(model, NIter, lr, batch_size=batch_size):\n", + " params = model.dnn.trainable_params()\n", + " optimizer_Adam = nn.Adam(params, learning_rate=lr)\n", + " grad_fn = ms.value_and_grad(model.loss_fn, None, optimizer_Adam.parameters, has_aux=True)\n", + " model.dnn.set_train()\n", + " N_data = model.t_f.shape[0]\n", + " start_time = time.time()\n", + " for epoch in range(1, 1+NIter):\n", + " idx_data_0 = np.random.choice(N_data, batch_size)\n", + " idx_data = Tensor(idx_data_0)\n", + " x_batch = model.x_f[idx_data, :]\n", + " y_batch = model.y_f[idx_data, :]\n", + " z_batch = model.z_f[idx_data, :]\n", + " t_batch = model.t_f[idx_data, :]\n", + " (loss, loss_b, loss_i, loss_f), grads = grad_fn(model.xb, model.yb, model.zb, model.tb, model.xi, model.yi, model.zi, model.ti, x_batch, y_batch, z_batch, t_batch, model.ub, model.vb, model.wb, model.ui, model.vi, model.wi)\n", + " optimizer_Adam(grads)\n", + " if epoch % 10 == 0:\n", + " elapsed = time.time() - start_time\n", + " print('It: %d, Total_loss: %.3e, Loss_b: %.3e, Loss_i: %.3e, Loss_f: %.3e, Time: %.2f' %\\\n", + " (epoch, loss.item(), loss_b.item(), loss_i.item(), loss_f.item(), elapsed))\n", + " global loss_history_Adam_Pretrain\n", + " global loss_b_history_Adam_Pretrain\n", + " global loss_i_history_Adam_Pretrain\n", + " global loss_f_history_Adam_Pretrain\n", + "\n", + " loss_history_Adam_Pretrain = np.append(loss_history_Adam_Pretrain, loss.numpy())\n", + " loss_b_history_Adam_Pretrain = np.append(loss_b_history_Adam_Pretrain, loss_b.numpy())\n", + " loss_i_history_Adam_Pretrain = np.append(loss_i_history_Adam_Pretrain, loss_i.numpy())\n", + " loss_f_history_Adam_Pretrain = np.append(loss_f_history_Adam_Pretrain, loss_f.numpy())\n", + "\n", + " start_time = time.time()" + ] + }, + { + "cell_type": "markdown", + "id": "cfedd05b-3d38-4156-b061-8278d232a7f9", + "metadata": {}, + "source": [ + "Run training and save the trained model." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c658395b-cade-408e-b999-585ba1ae73d5", + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "It: 0, Total_loss: 1.583e+02, Loss_b: 6.385e-01, Loss_i: 7.558e-01, Loss_f: 1.891e+01, Time: 14.63\n", + "It: 10, Total_loss: 1.339e+02, Loss_b: 5.654e-01, Loss_i: 6.433e-01, Loss_f: 1.302e+01, Time: 4.17\n", + "It: 20, Total_loss: 8.946e+01, Loss_b: 3.713e-01, Loss_i: 4.430e-01, Loss_f: 8.031e+00, Time: 4.22\n", + "It: 30, Total_loss: 6.822e+01, Loss_b: 3.025e-01, Loss_i: 2.891e-01, Loss_f: 9.061e+00, Time: 4.23\n", + "It: 40, Total_loss: 5.122e+01, Loss_b: 2.249e-01, Loss_i: 2.195e-01, Loss_f: 6.779e+00, Time: 4.23\n", + "It: 50, Total_loss: 4.095e+01, Loss_b: 1.834e-01, Loss_i: 1.549e-01, Loss_f: 7.122e+00, Time: 4.22\n", + "It: 60, Total_loss: 3.723e+01, Loss_b: 1.664e-01, Loss_i: 1.392e-01, Loss_f: 6.662e+00, Time: 4.25\n", + "It: 70, Total_loss: 3.123e+01, Loss_b: 1.457e-01, Loss_i: 1.092e-01, Loss_f: 5.742e+00, Time: 4.26\n", + "It: 80, Total_loss: 2.830e+01, Loss_b: 1.286e-01, Loss_i: 9.858e-02, Loss_f: 5.586e+00, Time: 4.23\n", + "It: 90, Total_loss: 2.403e+01, Loss_b: 1.046e-01, Loss_i: 7.816e-02, Loss_f: 5.755e+00, Time: 4.28\n", + "It: 100, Total_loss: 2.149e+01, Loss_b: 9.192e-02, Loss_i: 6.689e-02, Loss_f: 5.609e+00, Time: 4.27\n", + "It: 110, Total_loss: 1.913e+01, Loss_b: 8.201e-02, Loss_i: 5.710e-02, Loss_f: 5.223e+00, Time: 4.29\n", + "It: 120, Total_loss: 2.604e+01, Loss_b: 1.202e-01, Loss_i: 9.009e-02, Loss_f: 5.009e+00, Time: 4.31\n", + "It: 130, Total_loss: 1.589e+01, Loss_b: 6.606e-02, Loss_i: 4.568e-02, Loss_f: 4.713e+00, Time: 4.31\n", + "It: 140, Total_loss: 1.695e+01, Loss_b: 7.117e-02, Loss_i: 5.391e-02, Loss_f: 4.445e+00, Time: 4.29\n", + "It: 150, Total_loss: 1.529e+01, Loss_b: 6.290e-02, Loss_i: 4.427e-02, Loss_f: 4.573e+00, Time: 4.33\n", + "It: 160, Total_loss: 1.585e+01, Loss_b: 6.501e-02, Loss_i: 5.110e-02, Loss_f: 4.235e+00, Time: 4.31\n", + "It: 170, Total_loss: 1.325e+01, Loss_b: 5.349e-02, Loss_i: 3.818e-02, Loss_f: 4.081e+00, Time: 4.30\n", + "It: 180, Total_loss: 1.365e+01, Loss_b: 5.630e-02, Loss_i: 4.049e-02, Loss_f: 3.966e+00, Time: 4.37\n", + "It: 190, Total_loss: 1.153e+01, Loss_b: 4.635e-02, Loss_i: 3.206e-02, Loss_f: 3.685e+00, Time: 4.30\n", + "It: 200, Total_loss: 1.048e+01, Loss_b: 4.066e-02, Loss_i: 2.881e-02, Loss_f: 3.531e+00, Time: 4.35\n", + "It: 210, Total_loss: 9.349e+00, Loss_b: 3.611e-02, Loss_i: 2.546e-02, Loss_f: 3.193e+00, Time: 4.35\n", + "It: 220, Total_loss: 3.285e+01, Loss_b: 1.496e-01, Loss_i: 1.481e-01, Loss_f: 3.084e+00, Time: 4.36\n", + "It: 230, Total_loss: 1.309e+01, Loss_b: 5.685e-02, Loss_i: 4.612e-02, Loss_f: 2.790e+00, Time: 4.35\n", + "It: 240, Total_loss: 8.711e+00, Loss_b: 3.276e-02, Loss_i: 2.655e-02, Loss_f: 2.780e+00, Time: 4.37\n", + "It: 250, Total_loss: 8.350e+00, Loss_b: 3.038e-02, Loss_i: 2.387e-02, Loss_f: 2.925e+00, Time: 4.31\n", + "It: 260, Total_loss: 3.457e+01, Loss_b: 1.534e-01, Loss_i: 1.672e-01, Loss_f: 2.513e+00, Time: 4.39\n", + "It: 270, Total_loss: 1.192e+01, Loss_b: 5.266e-02, Loss_i: 3.971e-02, Loss_f: 2.680e+00, Time: 4.35\n", + "It: 280, Total_loss: 1.006e+01, Loss_b: 4.145e-02, Loss_i: 3.491e-02, Loss_f: 2.421e+00, Time: 4.33\n", + "It: 290, Total_loss: 6.854e+00, Loss_b: 2.660e-02, Loss_i: 1.906e-02, Loss_f: 2.288e+00, Time: 4.35\n", + "It: 300, Total_loss: 6.489e+00, Loss_b: 2.398e-02, Loss_i: 1.775e-02, Loss_f: 2.316e+00, Time: 4.31\n", + "It: 310, Total_loss: 5.934e+00, Loss_b: 2.192e-02, Loss_i: 1.608e-02, Loss_f: 2.135e+00, Time: 4.36\n", + "It: 320, Total_loss: 5.373e+00, Loss_b: 1.975e-02, Loss_i: 1.401e-02, Loss_f: 1.997e+00, Time: 4.32\n", + "It: 330, Total_loss: 2.589e+01, Loss_b: 1.093e-01, Loss_i: 1.278e-01, Loss_f: 2.179e+00, Time: 4.35\n", + "It: 340, Total_loss: 7.844e+00, Loss_b: 3.105e-02, Loss_i: 2.841e-02, Loss_f: 1.897e+00, Time: 4.33\n", + "It: 350, Total_loss: 5.789e+00, Loss_b: 2.054e-02, Loss_i: 1.715e-02, Loss_f: 2.020e+00, Time: 4.30\n", + "It: 360, Total_loss: 4.929e+00, Loss_b: 1.746e-02, Loss_i: 1.271e-02, Loss_f: 1.912e+00, Time: 4.35\n", + "It: 370, Total_loss: 4.540e+00, Loss_b: 1.605e-02, Loss_i: 1.103e-02, Loss_f: 1.832e+00, Time: 4.33\n", + "It: 380, Total_loss: 4.350e+00, Loss_b: 1.506e-02, Loss_i: 1.077e-02, Loss_f: 1.767e+00, Time: 4.34\n", + "It: 390, Total_loss: 4.076e+00, Loss_b: 1.387e-02, Loss_i: 9.670e-03, Loss_f: 1.721e+00, Time: 4.32\n", + "It: 400, Total_loss: 3.871e+00, Loss_b: 1.333e-02, Loss_i: 9.054e-03, Loss_f: 1.633e+00, Time: 4.35\n", + "It: 410, Total_loss: 3.836e+00, Loss_b: 1.356e-02, Loss_i: 9.326e-03, Loss_f: 1.547e+00, Time: 4.39\n", + "It: 420, Total_loss: 1.469e+01, Loss_b: 6.135e-02, Loss_i: 7.144e-02, Loss_f: 1.410e+00, Time: 4.35\n", + "It: 430, Total_loss: 9.486e+00, Loss_b: 3.899e-02, Loss_i: 4.083e-02, Loss_f: 1.503e+00, Time: 4.33\n", + "It: 440, Total_loss: 4.276e+00, Loss_b: 1.586e-02, Loss_i: 1.093e-02, Loss_f: 1.597e+00, Time: 4.31\n", + "It: 450, Total_loss: 4.369e+00, Loss_b: 1.655e-02, Loss_i: 1.267e-02, Loss_f: 1.447e+00, Time: 4.38\n", + "It: 460, Total_loss: 3.708e+00, Loss_b: 1.338e-02, Loss_i: 9.417e-03, Loss_f: 1.428e+00, Time: 4.35\n", + "It: 470, Total_loss: 3.409e+00, Loss_b: 1.175e-02, Loss_i: 7.869e-03, Loss_f: 1.447e+00, Time: 4.33\n", + "It: 480, Total_loss: 3.202e+00, Loss_b: 1.058e-02, Loss_i: 7.070e-03, Loss_f: 1.437e+00, Time: 4.30\n", + "It: 490, Total_loss: 3.011e+00, Loss_b: 1.008e-02, Loss_i: 6.602e-03, Loss_f: 1.342e+00, Time: 4.34\n", + "It: 500, Total_loss: 2.883e+00, Loss_b: 9.606e-03, Loss_i: 6.225e-03, Loss_f: 1.300e+00, Time: 4.43\n", + "It: 510, Total_loss: 3.084e+00, Loss_b: 1.055e-02, Loss_i: 7.329e-03, Loss_f: 1.296e+00, Time: 4.34\n", + "It: 520, Total_loss: 5.132e+00, Loss_b: 2.041e-02, Loss_i: 1.931e-02, Loss_f: 1.160e+00, Time: 4.37\n", + "It: 530, Total_loss: 4.985e+00, Loss_b: 1.992e-02, Loss_i: 1.770e-02, Loss_f: 1.223e+00, Time: 4.36\n", + "It: 540, Total_loss: 2.961e+00, Loss_b: 9.838e-03, Loss_i: 6.820e-03, Loss_f: 1.295e+00, Time: 4.35\n", + "It: 550, Total_loss: 2.722e+00, Loss_b: 9.219e-03, Loss_i: 6.062e-03, Loss_f: 1.194e+00, Time: 4.39\n", + "It: 560, Total_loss: 2.651e+00, Loss_b: 8.777e-03, Loss_i: 5.731e-03, Loss_f: 1.200e+00, Time: 4.34\n", + "It: 570, Total_loss: 2.435e+00, Loss_b: 8.007e-03, Loss_i: 5.153e-03, Loss_f: 1.119e+00, Time: 4.31\n", + "It: 580, Total_loss: 2.359e+00, Loss_b: 7.729e-03, Loss_i: 4.839e-03, Loss_f: 1.103e+00, Time: 4.37\n", + "It: 590, Total_loss: 2.411e+00, Loss_b: 7.893e-03, Loss_i: 5.292e-03, Loss_f: 1.093e+00, Time: 4.33\n", + "It: 600, Total_loss: 6.628e+00, Loss_b: 2.613e-02, Loss_i: 2.924e-02, Loss_f: 1.091e+00, Time: 4.33\n", + "It: 610, Total_loss: 3.092e+00, Loss_b: 1.098e-02, Loss_i: 9.677e-03, Loss_f: 1.026e+00, Time: 4.40\n", + "It: 620, Total_loss: 2.359e+00, Loss_b: 7.858e-03, Loss_i: 5.543e-03, Loss_f: 1.018e+00, Time: 4.35\n", + "It: 630, Total_loss: 2.168e+00, Loss_b: 7.162e-03, Loss_i: 4.739e-03, Loss_f: 9.775e-01, Time: 4.34\n", + "It: 640, Total_loss: 2.054e+00, Loss_b: 6.717e-03, Loss_i: 4.270e-03, Loss_f: 9.550e-01, Time: 4.35\n", + "It: 650, Total_loss: 1.949e+00, Loss_b: 6.251e-03, Loss_i: 3.827e-03, Loss_f: 9.413e-01, Time: 4.39\n", + "It: 660, Total_loss: 7.125e+00, Loss_b: 2.947e-02, Loss_i: 3.179e-02, Loss_f: 9.988e-01, Time: 4.35\n", + "It: 670, Total_loss: 8.039e+00, Loss_b: 3.376e-02, Loss_i: 3.641e-02, Loss_f: 1.022e+00, Time: 4.33\n", + "It: 680, Total_loss: 2.239e+00, Loss_b: 7.544e-03, Loss_i: 5.005e-03, Loss_f: 9.844e-01, Time: 4.38\n", + "It: 690, Total_loss: 2.498e+00, Loss_b: 8.721e-03, Loss_i: 7.084e-03, Loss_f: 9.173e-01, Time: 4.37\n", + "It: 700, Total_loss: 2.013e+00, Loss_b: 6.654e-03, Loss_i: 4.436e-03, Loss_f: 9.038e-01, Time: 4.32\n", + "It: 710, Total_loss: 1.930e+00, Loss_b: 6.223e-03, Loss_i: 3.988e-03, Loss_f: 9.092e-01, Time: 4.34\n", + "It: 720, Total_loss: 1.806e+00, Loss_b: 5.569e-03, Loss_i: 3.448e-03, Loss_f: 9.047e-01, Time: 4.42\n", + "It: 730, Total_loss: 1.723e+00, Loss_b: 5.383e-03, Loss_i: 3.242e-03, Loss_f: 8.601e-01, Time: 4.34\n", + "It: 740, Total_loss: 3.905e+00, Loss_b: 1.531e-02, Loss_i: 1.506e-02, Loss_f: 8.681e-01, Time: 4.36\n", + "It: 750, Total_loss: 2.988e+00, Loss_b: 1.194e-02, Loss_i: 1.040e-02, Loss_f: 7.538e-01, Time: 4.35\n", + "It: 760, Total_loss: 2.161e+00, Loss_b: 7.457e-03, Loss_i: 5.830e-03, Loss_f: 8.320e-01, Time: 4.34\n", + "It: 770, Total_loss: 1.972e+00, Loss_b: 6.726e-03, Loss_i: 4.789e-03, Loss_f: 8.208e-01, Time: 4.34\n", + "It: 780, Total_loss: 1.701e+00, Loss_b: 5.803e-03, Loss_i: 3.587e-03, Loss_f: 7.619e-01, Time: 4.29\n", + "It: 790, Total_loss: 1.591e+00, Loss_b: 5.103e-03, Loss_i: 3.114e-03, Loss_f: 7.698e-01, Time: 4.36\n", + "It: 800, Total_loss: 1.535e+00, Loss_b: 4.848e-03, Loss_i: 2.912e-03, Loss_f: 7.591e-01, Time: 4.43\n", + "It: 810, Total_loss: 1.466e+01, Loss_b: 6.288e-02, Loss_i: 7.408e-02, Loss_f: 9.674e-01, Time: 4.37\n", + "It: 820, Total_loss: 4.528e+00, Loss_b: 1.733e-02, Loss_i: 1.952e-02, Loss_f: 8.425e-01, Time: 4.42\n", + "It: 830, Total_loss: 2.498e+00, Loss_b: 9.084e-03, Loss_i: 8.434e-03, Loss_f: 7.457e-01, Time: 4.38\n", + "It: 840, Total_loss: 1.777e+00, Loss_b: 5.724e-03, Loss_i: 4.544e-03, Loss_f: 7.503e-01, Time: 4.41\n", + "It: 850, Total_loss: 1.575e+00, Loss_b: 5.009e-03, Loss_i: 3.406e-03, Loss_f: 7.332e-01, Time: 4.35\n", + "It: 860, Total_loss: 1.418e+00, Loss_b: 4.292e-03, Loss_i: 2.733e-03, Loss_f: 7.161e-01, Time: 4.38\n", + "It: 870, Total_loss: 1.320e+00, Loss_b: 4.021e-03, Loss_i: 2.429e-03, Loss_f: 6.748e-01, Time: 4.34\n", + "It: 880, Total_loss: 1.290e+00, Loss_b: 3.938e-03, Loss_i: 2.351e-03, Loss_f: 6.612e-01, Time: 4.34\n", + "It: 890, Total_loss: 1.261e+00, Loss_b: 3.838e-03, Loss_i: 2.264e-03, Loss_f: 6.513e-01, Time: 4.32\n", + "It: 900, Total_loss: 1.255e+00, Loss_b: 3.763e-03, Loss_i: 2.241e-03, Loss_f: 6.545e-01, Time: 4.34\n", + "It: 910, Total_loss: 1.232e+00, Loss_b: 3.749e-03, Loss_i: 2.343e-03, Loss_f: 6.233e-01, Time: 4.33\n", + "It: 920, Total_loss: 6.618e+00, Loss_b: 2.682e-02, Loss_i: 3.284e-02, Loss_f: 6.520e-01, Time: 4.34\n", + "It: 930, Total_loss: 2.011e+00, Loss_b: 7.589e-03, Loss_i: 6.499e-03, Loss_f: 6.022e-01, Time: 4.39\n", + "It: 940, Total_loss: 1.656e+00, Loss_b: 5.748e-03, Loss_i: 4.692e-03, Loss_f: 6.119e-01, Time: 4.34\n", + "It: 950, Total_loss: 1.524e+00, Loss_b: 5.275e-03, Loss_i: 4.060e-03, Loss_f: 5.907e-01, Time: 4.34\n", + "It: 960, Total_loss: 1.150e+00, Loss_b: 3.567e-03, Loss_i: 2.078e-03, Loss_f: 5.854e-01, Time: 4.36\n", + "It: 970, Total_loss: 1.136e+00, Loss_b: 3.517e-03, Loss_i: 2.214e-03, Loss_f: 5.624e-01, Time: 4.40\n", + "It: 980, Total_loss: 1.098e+00, Loss_b: 3.371e-03, Loss_i: 2.037e-03, Loss_f: 5.576e-01, Time: 4.31\n", + "It: 990, Total_loss: 1.058e+00, Loss_b: 3.218e-03, Loss_i: 1.868e-03, Loss_f: 5.492e-01, Time: 4.34\n", + "It: 1000, Total_loss: 1.048e+00, Loss_b: 3.260e-03, Loss_i: 1.985e-03, Loss_f: 5.237e-01, Time: 4.35\n", + "It: 1010, Total_loss: 9.346e+00, Loss_b: 3.993e-02, Loss_i: 4.750e-02, Loss_f: 6.030e-01, Time: 4.38\n", + "It: 1020, Total_loss: 5.139e+00, Loss_b: 2.211e-02, Loss_i: 2.336e-02, Loss_f: 5.921e-01, Time: 4.34\n", + "It: 1030, Total_loss: 2.316e+00, Loss_b: 8.761e-03, Loss_i: 8.724e-03, Loss_f: 5.671e-01, Time: 4.33\n", + "It: 1040, Total_loss: 1.314e+00, Loss_b: 4.416e-03, Loss_i: 3.425e-03, Loss_f: 5.302e-01, Time: 4.34\n", + "It: 1050, Total_loss: 1.101e+00, Loss_b: 3.623e-03, Loss_i: 2.288e-03, Loss_f: 5.101e-01, Time: 4.35\n", + "It: 1060, Total_loss: 9.831e-01, Loss_b: 2.902e-03, Loss_i: 1.671e-03, Loss_f: 5.259e-01, Time: 4.31\n", + "It: 1070, Total_loss: 9.554e-01, Loss_b: 2.894e-03, Loss_i: 1.657e-03, Loss_f: 5.003e-01, Time: 4.33\n", + "It: 1080, Total_loss: 9.228e-01, Loss_b: 2.796e-03, Loss_i: 1.600e-03, Loss_f: 4.832e-01, Time: 4.33\n", + "It: 1090, Total_loss: 9.017e-01, Loss_b: 2.708e-03, Loss_i: 1.540e-03, Loss_f: 4.769e-01, Time: 4.35\n", + "It: 1100, Total_loss: 9.491e-01, Loss_b: 2.927e-03, Loss_i: 1.858e-03, Loss_f: 4.705e-01, Time: 4.37\n", + "It: 1110, Total_loss: 6.445e+00, Loss_b: 2.654e-02, Loss_i: 3.269e-02, Loss_f: 5.224e-01, Time: 4.38\n", + "It: 1120, Total_loss: 5.876e+00, Loss_b: 2.427e-02, Loss_i: 2.891e-02, Loss_f: 5.584e-01, Time: 4.34\n", + "It: 1130, Total_loss: 1.434e+00, Loss_b: 5.268e-03, Loss_i: 4.353e-03, Loss_f: 4.723e-01, Time: 4.35\n" + ] + } + ], + "source": [ + "for epoch, lr in zip(epochs, learning_rate):\n", + " train(model, int(epoch), lr, batch_size=batch_size)\n", + "ms.save_checkpoint(model.dnn, f'model/{second_path}/model.ckpt')" + ] + }, + { + "cell_type": "markdown", + "id": "7032afa7-b415-4ef1-b49f-97a22e6ba634", + "metadata": {}, + "source": [ + "Save the loss function" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "9e56141a-8ec1-4d55-9f4e-cea19c209e8f", + "metadata": {}, + "outputs": [], + "source": [ + "np.save(f'Loss-Coe/{second_path}/loss_history_Adam_Pretrain', loss_history_Adam_Pretrain)\n", + "np.save(f'Loss-Coe/{second_path}/loss_b_history_Adam_Pretrain', loss_b_history_Adam_Pretrain)\n", + "np.save(f'Loss-Coe/{second_path}/loss_i_history_Adam_Pretrain', loss_i_history_Adam_Pretrain)\n", + "np.save(f'Loss-Coe/{second_path}/loss_f_history_Adam_Pretrain', loss_f_history_Adam_Pretrain)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.19" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/MindFlow/applications/research/NSFNets/NSFNets_CN.ipynb b/MindFlow/applications/research/NSFNets/NSFNets_CN.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..5de07992284ee0eab635aaf6f8039c0994e27c85 --- /dev/null +++ b/MindFlow/applications/research/NSFNets/NSFNets_CN.ipynb @@ -0,0 +1,426 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "de7a09e7-32fc-4059-9478-6c6fc6667f57", + "metadata": {}, + "source": [ + "# NSFNets: 用于不可压缩 Navier-Stokes 方程求解的物理信息神经网络\n", + "\n", + "## 环境安装\n", + "\n", + "本案例要求 MindSpore >= 2.0.0 版本以调用如下接口: mindspore.jit, mindspore.jit_class, mindspore.data_sink。具体请查看MindSpore安装。\n", + "\n", + "此外，你需要安装 MindFlow >=0.1.0 版本。如果当前环境还没有安装，请按照下列方式选择后端和版本进行安装。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5a449343-731b-46bb-b8dd-f7a8c1ba8071", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[33mWARNING: Skipping mindflow-gpu as it is not installed.\u001b[0m\u001b[33m\n", + "\u001b[0m" + ] + } + ], + "source": [ + "mindflow_version = \"0.1.0\" # update if needed\n", + "# GPU Comment out the following code if you are using NPU.\n", + "!pip uninstall -y mindflow-gpu\n", + "!pip install mindflow-gpu==$mindflow_version\n", + "\n", + "# NPU Uncomment if needed.\n", + "# !pip uninstall -y mindflow-ascend\n", + "# !pip install mindflow-ascend==$mindflow_version" + ] + }, + { + "cell_type": "markdown", + "id": "25fe4081-9d91-44c1-b7de-b50a7645e090", + "metadata": {}, + "source": [ + "## 背景介绍\n", + "\n", + "在AI-CFD的工业应用方面，对NavierStokes方程的求解以及对多精度数据的融合的场景十分广泛，具有重要的经济和社会意义。求解不适定问题（例如部分边界条件缺失）或反演问题是其中的重点和难点之一，且往往代价高昂，需要推导适用于特定问题的不同公式、编写全新代码。如何用一套统一代码以相同计算代价解决上述问题亟需深入研究。在此，金晓威和李惠等使用物理信息神经网络（PINNs），通过自动微分将控制方程直接编码到深度神经网络中，以克服上述一些模拟不可压缩层流和湍流流动的限制。并开发了Navier-Stokes流动网络（NSFnets，Navier-Stokes flow nets）。" + ] + }, + { + "cell_type": "markdown", + "id": "50540e43-3c8e-45f0-8e2b-78831ec89684", + "metadata": {}, + "source": [ + "## 模型框架\n", + "\n", + "模型框架图如下所示:\n", + "\n", + "![NSFNet](images/NSFNet.png)\n", + "\n", + "图中表示的是压力-速度形式的纳维-斯托克斯方程求解网络，利用神经网络的自动微分计算方程中所需要的偏导项，损失函数包括，边界条件损失，初始条件损失以及为了满足方程平衡的物理损失。\n", + "\n", + "## 准备环节\n", + "\n", + "实践前，确保已经正确安装合适版本的MindSpore。如果没有，可以通过：\n", + "\n", + "* [MindSpore安装页面](https://www.mindspore.cn/install) 安装MindSpore。\n", + "\n", + "## 数据集的准备\n", + "\n", + "数据集可以通过提供的代码直接生成。" + ] + }, + { + "cell_type": "markdown", + "id": "23990483-d613-4c8b-aa7e-0a8b16a89b13", + "metadata": {}, + "source": [ + "## 模型训练\n", + "\n", + "引入代码包" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "e3d17b8a-bde1-4ccd-9345-ccfaa2cd1bfe", + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "import argparse\n", + "\n", + "import time\n", + "import numpy as np\n", + "import mindspore as ms\n", + "from mindspore import set_seed, context, nn, Tensor\n", + "from src.network import VPNSFNets\n", + "from src.datasets import read_training_data" + ] + }, + { + "cell_type": "markdown", + "id": "0b8e1d64-580b-4ae9-a7f6-164b0ee0adf3", + "metadata": {}, + "source": [ + "模型相关参数的设置以及训练模型的定义" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "8a84c0ff-d4cf-41dd-8afc-db8f234d2108", + "metadata": {}, + "outputs": [], + "source": [ + "parser = argparse.ArgumentParser()\n", + "parser.add_argument('--model_name', type=str, default='NSFNet')\n", + "parser.add_argument('--case', type=str, default='Three dimensional Beltrami flow')\n", + "parser.add_argument('--device', type=str, default='GPU')\n", + "parser.add_argument('--device_id', type=str, default='1')\n", + "parser.add_argument('--load_params', type=str, default=False)\n", + "parser.add_argument('--second_path', type=str, default='train')\n", + "parser.add_argument('--network_size', type=int, default=[4] + 10 * [100 * 1] + [4])\n", + "parser.add_argument('--learning_rate', type=int, default=[1.0e-03, 1.0e-04, 1.0e-05, 1.0e-06])\n", + "parser.add_argument('--epochs', type=int, default=[5e3, 5e3, 5e4, 5e4])\n", + "parser.add_argument('--batch_size', type=int, default=10000)\n", + "parser.add_argument('--re', type=int, default=1)\n", + "args = parser.parse_known_args()[0]\n", + "# args = parser.parse_args()\n", + "\n", + "model_name = args.model_name\n", + "case = args.case\n", + "device = args.device\n", + "device_id = args.device_id\n", + "network_size = args.network_size\n", + "learning_rate = args.learning_rate\n", + "epochs = args.epochs\n", + "batch_size = args.batch_size\n", + "load_params = args.load_params\n", + "second_path = args.second_path\n", + "re = args.re\n", + "\n", + "use_ascend = context.get_context(attr_key='device_target') == \"Ascend\"\n", + "\n", + "if use_ascend:\n", + " msfloat_type = ms.float16\n", + " npfloat_type = np.float16\n", + "else:\n", + " msfloat_type = ms.float32\n", + " npfloat_type = np.float32\n", + "\n", + "os.environ['CUDA_VISIBLE_DEVICES'] = device_id\n", + "context.set_context(mode=context.GRAPH_MODE, save_graphs=False, device_target=device)\n", + "\n", + "x_b, y_b, z_b, t_b, x_i, y_i, z_i, t_i, u_b, v_b, w_b, u_i, v_i, w_i, x_f, y_f, z_f, t_f, X_min, X_max = read_training_data()\n", + "\n", + "model = VPNSFNets(x_b, y_b, z_b, t_b, x_i, y_i, z_i, t_i, u_b, v_b, w_b, u_i, v_i, w_i, x_f, y_f, z_f, t_f, network_size, re, \\\n", + " X_min, X_max, use_ascend, msfloat_type, npfloat_type, load_params, second_path)" + ] + }, + { + "cell_type": "markdown", + "id": "387cb210-078e-41e1-b960-e432af9c8d5f", + "metadata": {}, + "source": [ + "记录损失变化" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "482e625a-262a-4f06-a4a7-834766ed6ab4", + "metadata": {}, + "outputs": [], + "source": [ + "# Adam loss history\n", + "loss_history_Adam_Pretrain = np.empty([0])\n", + "loss_b_history_Adam_Pretrain = np.empty([0])\n", + "loss_i_history_Adam_Pretrain = np.empty([0])\n", + "loss_f_history_Adam_Pretrain = np.empty([0])\n", + "np.random.seed(123456)\n", + "set_seed(123456)" + ] + }, + { + "cell_type": "markdown", + "id": "30f8c191-ea21-4055-9d32-dfae30c96a33", + "metadata": {}, + "source": [ + "代码训练与输出结果部分" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "cff1733f-6e6a-47ff-928f-be3a348e98da", + "metadata": {}, + "outputs": [], + "source": [ + "def train(model, NIter, lr, batch_size=batch_size):\n", + " params = model.dnn.trainable_params()\n", + " optimizer_Adam = nn.Adam(params, learning_rate=lr)\n", + " grad_fn = ms.value_and_grad(model.loss_fn, None, optimizer_Adam.parameters, has_aux=True)\n", + " model.dnn.set_train()\n", + " N_data = model.t_f.shape[0]\n", + " start_time = time.time()\n", + "\n", + " for epoch in range(1, 1+NIter):\n", + " idx_data_0 = np.random.choice(N_data, batch_size)\n", + " idx_data = Tensor(idx_data_0)\n", + " x_batch = model.x_f[idx_data, :]\n", + " y_batch = model.y_f[idx_data, :]\n", + " z_batch = model.z_f[idx_data, :]\n", + " t_batch = model.t_f[idx_data, :]\n", + " (loss, loss_b, loss_i, loss_f), grads = grad_fn(model.xb, model.yb, model.zb, model.tb, model.xi, model.yi, model.zi, model.ti, x_batch, y_batch, z_batch, t_batch, model.ub, model.vb, model.wb, model.ui, model.vi, model.wi)\n", + " optimizer_Adam(grads)\n", + "\n", + " if epoch % 10 == 0:\n", + " elapsed = time.time() - start_time\n", + " print('It: %d, Total_loss: %.3e, Loss_b: %.3e, Loss_i: %.3e, Loss_f: %.3e, Time: %.2f' %\\\n", + " (epoch, loss.item(), loss_b.item(), loss_i.item(), loss_f.item(), elapsed))\n", + "\n", + " global loss_history_Adam_Pretrain\n", + " global loss_b_history_Adam_Pretrain\n", + " global loss_i_history_Adam_Pretrain\n", + " global loss_f_history_Adam_Pretrain\n", + "\n", + " loss_history_Adam_Pretrain = np.append(loss_history_Adam_Pretrain, loss.numpy())\n", + " loss_b_history_Adam_Pretrain = np.append(loss_b_history_Adam_Pretrain, loss_b.numpy())\n", + " loss_i_history_Adam_Pretrain = np.append(loss_i_history_Adam_Pretrain, loss_i.numpy())\n", + " loss_f_history_Adam_Pretrain = np.append(loss_f_history_Adam_Pretrain, loss_f.numpy())\n", + "\n", + " start_time = time.time()" + ] + }, + { + "cell_type": "markdown", + "id": "cfedd05b-3d38-4156-b061-8278d232a7f9", + "metadata": {}, + "source": [ + "运行训练与保存训练模型" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c658395b-cade-408e-b999-585ba1ae73d5", + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "It: 0, Total_loss: 1.583e+02, Loss_b: 6.385e-01, Loss_i: 7.558e-01, Loss_f: 1.891e+01, Time: 14.63\n", + "It: 10, Total_loss: 1.339e+02, Loss_b: 5.654e-01, Loss_i: 6.433e-01, Loss_f: 1.302e+01, Time: 4.17\n", + "It: 20, Total_loss: 8.946e+01, Loss_b: 3.713e-01, Loss_i: 4.430e-01, Loss_f: 8.031e+00, Time: 4.22\n", + "It: 30, Total_loss: 6.822e+01, Loss_b: 3.025e-01, Loss_i: 2.891e-01, Loss_f: 9.061e+00, Time: 4.23\n", + "It: 40, Total_loss: 5.122e+01, Loss_b: 2.249e-01, Loss_i: 2.195e-01, Loss_f: 6.779e+00, Time: 4.23\n", + "It: 50, Total_loss: 4.095e+01, Loss_b: 1.834e-01, Loss_i: 1.549e-01, Loss_f: 7.122e+00, Time: 4.22\n", + "It: 60, Total_loss: 3.723e+01, Loss_b: 1.664e-01, Loss_i: 1.392e-01, Loss_f: 6.662e+00, Time: 4.25\n", + "It: 70, Total_loss: 3.123e+01, Loss_b: 1.457e-01, Loss_i: 1.092e-01, Loss_f: 5.742e+00, Time: 4.26\n", + "It: 80, Total_loss: 2.830e+01, Loss_b: 1.286e-01, Loss_i: 9.858e-02, Loss_f: 5.586e+00, Time: 4.23\n", + "It: 90, Total_loss: 2.403e+01, Loss_b: 1.046e-01, Loss_i: 7.816e-02, Loss_f: 5.755e+00, Time: 4.28\n", + "It: 100, Total_loss: 2.149e+01, Loss_b: 9.192e-02, Loss_i: 6.689e-02, Loss_f: 5.609e+00, Time: 4.27\n", + "It: 110, Total_loss: 1.913e+01, Loss_b: 8.201e-02, Loss_i: 5.710e-02, Loss_f: 5.223e+00, Time: 4.29\n", + "It: 120, Total_loss: 2.604e+01, Loss_b: 1.202e-01, Loss_i: 9.009e-02, Loss_f: 5.009e+00, Time: 4.31\n", + "It: 130, Total_loss: 1.589e+01, Loss_b: 6.606e-02, Loss_i: 4.568e-02, Loss_f: 4.713e+00, Time: 4.31\n", + "It: 140, Total_loss: 1.695e+01, Loss_b: 7.117e-02, Loss_i: 5.391e-02, Loss_f: 4.445e+00, Time: 4.29\n", + "It: 150, Total_loss: 1.529e+01, Loss_b: 6.290e-02, Loss_i: 4.427e-02, Loss_f: 4.573e+00, Time: 4.33\n", + "It: 160, Total_loss: 1.585e+01, Loss_b: 6.501e-02, Loss_i: 5.110e-02, Loss_f: 4.235e+00, Time: 4.31\n", + "It: 170, Total_loss: 1.325e+01, Loss_b: 5.349e-02, Loss_i: 3.818e-02, Loss_f: 4.081e+00, Time: 4.30\n", + "It: 180, Total_loss: 1.365e+01, Loss_b: 5.630e-02, Loss_i: 4.049e-02, Loss_f: 3.966e+00, Time: 4.37\n", + "It: 190, Total_loss: 1.153e+01, Loss_b: 4.635e-02, Loss_i: 3.206e-02, Loss_f: 3.685e+00, Time: 4.30\n", + "It: 200, Total_loss: 1.048e+01, Loss_b: 4.066e-02, Loss_i: 2.881e-02, Loss_f: 3.531e+00, Time: 4.35\n", + "It: 210, Total_loss: 9.349e+00, Loss_b: 3.611e-02, Loss_i: 2.546e-02, Loss_f: 3.193e+00, Time: 4.35\n", + "It: 220, Total_loss: 3.285e+01, Loss_b: 1.496e-01, Loss_i: 1.481e-01, Loss_f: 3.084e+00, Time: 4.36\n", + "It: 230, Total_loss: 1.309e+01, Loss_b: 5.685e-02, Loss_i: 4.612e-02, Loss_f: 2.790e+00, Time: 4.35\n", + "It: 240, Total_loss: 8.711e+00, Loss_b: 3.276e-02, Loss_i: 2.655e-02, Loss_f: 2.780e+00, Time: 4.37\n", + "It: 250, Total_loss: 8.350e+00, Loss_b: 3.038e-02, Loss_i: 2.387e-02, Loss_f: 2.925e+00, Time: 4.31\n", + "It: 260, Total_loss: 3.457e+01, Loss_b: 1.534e-01, Loss_i: 1.672e-01, Loss_f: 2.513e+00, Time: 4.39\n", + "It: 270, Total_loss: 1.192e+01, Loss_b: 5.266e-02, Loss_i: 3.971e-02, Loss_f: 2.680e+00, Time: 4.35\n", + "It: 280, Total_loss: 1.006e+01, Loss_b: 4.145e-02, Loss_i: 3.491e-02, Loss_f: 2.421e+00, Time: 4.33\n", + "It: 290, Total_loss: 6.854e+00, Loss_b: 2.660e-02, Loss_i: 1.906e-02, Loss_f: 2.288e+00, Time: 4.35\n", + "It: 300, Total_loss: 6.489e+00, Loss_b: 2.398e-02, Loss_i: 1.775e-02, Loss_f: 2.316e+00, Time: 4.31\n", + "It: 310, Total_loss: 5.934e+00, Loss_b: 2.192e-02, Loss_i: 1.608e-02, Loss_f: 2.135e+00, Time: 4.36\n", + "It: 320, Total_loss: 5.373e+00, Loss_b: 1.975e-02, Loss_i: 1.401e-02, Loss_f: 1.997e+00, Time: 4.32\n", + "It: 330, Total_loss: 2.589e+01, Loss_b: 1.093e-01, Loss_i: 1.278e-01, Loss_f: 2.179e+00, Time: 4.35\n", + "It: 340, Total_loss: 7.844e+00, Loss_b: 3.105e-02, Loss_i: 2.841e-02, Loss_f: 1.897e+00, Time: 4.33\n", + "It: 350, Total_loss: 5.789e+00, Loss_b: 2.054e-02, Loss_i: 1.715e-02, Loss_f: 2.020e+00, Time: 4.30\n", + "It: 360, Total_loss: 4.929e+00, Loss_b: 1.746e-02, Loss_i: 1.271e-02, Loss_f: 1.912e+00, Time: 4.35\n", + "It: 370, Total_loss: 4.540e+00, Loss_b: 1.605e-02, Loss_i: 1.103e-02, Loss_f: 1.832e+00, Time: 4.33\n", + "It: 380, Total_loss: 4.350e+00, Loss_b: 1.506e-02, Loss_i: 1.077e-02, Loss_f: 1.767e+00, Time: 4.34\n", + "It: 390, Total_loss: 4.076e+00, Loss_b: 1.387e-02, Loss_i: 9.670e-03, Loss_f: 1.721e+00, Time: 4.32\n", + "It: 400, Total_loss: 3.871e+00, Loss_b: 1.333e-02, Loss_i: 9.054e-03, Loss_f: 1.633e+00, Time: 4.35\n", + "It: 410, Total_loss: 3.836e+00, Loss_b: 1.356e-02, Loss_i: 9.326e-03, Loss_f: 1.547e+00, Time: 4.39\n", + "It: 420, Total_loss: 1.469e+01, Loss_b: 6.135e-02, Loss_i: 7.144e-02, Loss_f: 1.410e+00, Time: 4.35\n", + "It: 430, Total_loss: 9.486e+00, Loss_b: 3.899e-02, Loss_i: 4.083e-02, Loss_f: 1.503e+00, Time: 4.33\n", + "It: 440, Total_loss: 4.276e+00, Loss_b: 1.586e-02, Loss_i: 1.093e-02, Loss_f: 1.597e+00, Time: 4.31\n", + "It: 450, Total_loss: 4.369e+00, Loss_b: 1.655e-02, Loss_i: 1.267e-02, Loss_f: 1.447e+00, Time: 4.38\n", + "It: 460, Total_loss: 3.708e+00, Loss_b: 1.338e-02, Loss_i: 9.417e-03, Loss_f: 1.428e+00, Time: 4.35\n", + "It: 470, Total_loss: 3.409e+00, Loss_b: 1.175e-02, Loss_i: 7.869e-03, Loss_f: 1.447e+00, Time: 4.33\n", + "It: 480, Total_loss: 3.202e+00, Loss_b: 1.058e-02, Loss_i: 7.070e-03, Loss_f: 1.437e+00, Time: 4.30\n", + "It: 490, Total_loss: 3.011e+00, Loss_b: 1.008e-02, Loss_i: 6.602e-03, Loss_f: 1.342e+00, Time: 4.34\n", + "It: 500, Total_loss: 2.883e+00, Loss_b: 9.606e-03, Loss_i: 6.225e-03, Loss_f: 1.300e+00, Time: 4.43\n", + "It: 510, Total_loss: 3.084e+00, Loss_b: 1.055e-02, Loss_i: 7.329e-03, Loss_f: 1.296e+00, Time: 4.34\n", + "It: 520, Total_loss: 5.132e+00, Loss_b: 2.041e-02, Loss_i: 1.931e-02, Loss_f: 1.160e+00, Time: 4.37\n", + "It: 530, Total_loss: 4.985e+00, Loss_b: 1.992e-02, Loss_i: 1.770e-02, Loss_f: 1.223e+00, Time: 4.36\n", + "It: 540, Total_loss: 2.961e+00, Loss_b: 9.838e-03, Loss_i: 6.820e-03, Loss_f: 1.295e+00, Time: 4.35\n", + "It: 550, Total_loss: 2.722e+00, Loss_b: 9.219e-03, Loss_i: 6.062e-03, Loss_f: 1.194e+00, Time: 4.39\n", + "It: 560, Total_loss: 2.651e+00, Loss_b: 8.777e-03, Loss_i: 5.731e-03, Loss_f: 1.200e+00, Time: 4.34\n", + "It: 570, Total_loss: 2.435e+00, Loss_b: 8.007e-03, Loss_i: 5.153e-03, Loss_f: 1.119e+00, Time: 4.31\n", + "It: 580, Total_loss: 2.359e+00, Loss_b: 7.729e-03, Loss_i: 4.839e-03, Loss_f: 1.103e+00, Time: 4.37\n", + "It: 590, Total_loss: 2.411e+00, Loss_b: 7.893e-03, Loss_i: 5.292e-03, Loss_f: 1.093e+00, Time: 4.33\n", + "It: 600, Total_loss: 6.628e+00, Loss_b: 2.613e-02, Loss_i: 2.924e-02, Loss_f: 1.091e+00, Time: 4.33\n", + "It: 610, Total_loss: 3.092e+00, Loss_b: 1.098e-02, Loss_i: 9.677e-03, Loss_f: 1.026e+00, Time: 4.40\n", + "It: 620, Total_loss: 2.359e+00, Loss_b: 7.858e-03, Loss_i: 5.543e-03, Loss_f: 1.018e+00, Time: 4.35\n", + "It: 630, Total_loss: 2.168e+00, Loss_b: 7.162e-03, Loss_i: 4.739e-03, Loss_f: 9.775e-01, Time: 4.34\n", + "It: 640, Total_loss: 2.054e+00, Loss_b: 6.717e-03, Loss_i: 4.270e-03, Loss_f: 9.550e-01, Time: 4.35\n", + "It: 650, Total_loss: 1.949e+00, Loss_b: 6.251e-03, Loss_i: 3.827e-03, Loss_f: 9.413e-01, Time: 4.39\n", + "It: 660, Total_loss: 7.125e+00, Loss_b: 2.947e-02, Loss_i: 3.179e-02, Loss_f: 9.988e-01, Time: 4.35\n", + "It: 670, Total_loss: 8.039e+00, Loss_b: 3.376e-02, Loss_i: 3.641e-02, Loss_f: 1.022e+00, Time: 4.33\n", + "It: 680, Total_loss: 2.239e+00, Loss_b: 7.544e-03, Loss_i: 5.005e-03, Loss_f: 9.844e-01, Time: 4.38\n", + "It: 690, Total_loss: 2.498e+00, Loss_b: 8.721e-03, Loss_i: 7.084e-03, Loss_f: 9.173e-01, Time: 4.37\n", + "It: 700, Total_loss: 2.013e+00, Loss_b: 6.654e-03, Loss_i: 4.436e-03, Loss_f: 9.038e-01, Time: 4.32\n", + "It: 710, Total_loss: 1.930e+00, Loss_b: 6.223e-03, Loss_i: 3.988e-03, Loss_f: 9.092e-01, Time: 4.34\n", + "It: 720, Total_loss: 1.806e+00, Loss_b: 5.569e-03, Loss_i: 3.448e-03, Loss_f: 9.047e-01, Time: 4.42\n", + "It: 730, Total_loss: 1.723e+00, Loss_b: 5.383e-03, Loss_i: 3.242e-03, Loss_f: 8.601e-01, Time: 4.34\n", + "It: 740, Total_loss: 3.905e+00, Loss_b: 1.531e-02, Loss_i: 1.506e-02, Loss_f: 8.681e-01, Time: 4.36\n", + "It: 750, Total_loss: 2.988e+00, Loss_b: 1.194e-02, Loss_i: 1.040e-02, Loss_f: 7.538e-01, Time: 4.35\n", + "It: 760, Total_loss: 2.161e+00, Loss_b: 7.457e-03, Loss_i: 5.830e-03, Loss_f: 8.320e-01, Time: 4.34\n", + "It: 770, Total_loss: 1.972e+00, Loss_b: 6.726e-03, Loss_i: 4.789e-03, Loss_f: 8.208e-01, Time: 4.34\n", + "It: 780, Total_loss: 1.701e+00, Loss_b: 5.803e-03, Loss_i: 3.587e-03, Loss_f: 7.619e-01, Time: 4.29\n", + "It: 790, Total_loss: 1.591e+00, Loss_b: 5.103e-03, Loss_i: 3.114e-03, Loss_f: 7.698e-01, Time: 4.36\n", + "It: 800, Total_loss: 1.535e+00, Loss_b: 4.848e-03, Loss_i: 2.912e-03, Loss_f: 7.591e-01, Time: 4.43\n", + "It: 810, Total_loss: 1.466e+01, Loss_b: 6.288e-02, Loss_i: 7.408e-02, Loss_f: 9.674e-01, Time: 4.37\n", + "It: 820, Total_loss: 4.528e+00, Loss_b: 1.733e-02, Loss_i: 1.952e-02, Loss_f: 8.425e-01, Time: 4.42\n", + "It: 830, Total_loss: 2.498e+00, Loss_b: 9.084e-03, Loss_i: 8.434e-03, Loss_f: 7.457e-01, Time: 4.38\n", + "It: 840, Total_loss: 1.777e+00, Loss_b: 5.724e-03, Loss_i: 4.544e-03, Loss_f: 7.503e-01, Time: 4.41\n", + "It: 850, Total_loss: 1.575e+00, Loss_b: 5.009e-03, Loss_i: 3.406e-03, Loss_f: 7.332e-01, Time: 4.35\n", + "It: 860, Total_loss: 1.418e+00, Loss_b: 4.292e-03, Loss_i: 2.733e-03, Loss_f: 7.161e-01, Time: 4.38\n", + "It: 870, Total_loss: 1.320e+00, Loss_b: 4.021e-03, Loss_i: 2.429e-03, Loss_f: 6.748e-01, Time: 4.34\n", + "It: 880, Total_loss: 1.290e+00, Loss_b: 3.938e-03, Loss_i: 2.351e-03, Loss_f: 6.612e-01, Time: 4.34\n", + "It: 890, Total_loss: 1.261e+00, Loss_b: 3.838e-03, Loss_i: 2.264e-03, Loss_f: 6.513e-01, Time: 4.32\n", + "It: 900, Total_loss: 1.255e+00, Loss_b: 3.763e-03, Loss_i: 2.241e-03, Loss_f: 6.545e-01, Time: 4.34\n", + "It: 910, Total_loss: 1.232e+00, Loss_b: 3.749e-03, Loss_i: 2.343e-03, Loss_f: 6.233e-01, Time: 4.33\n", + "It: 920, Total_loss: 6.618e+00, Loss_b: 2.682e-02, Loss_i: 3.284e-02, Loss_f: 6.520e-01, Time: 4.34\n", + "It: 930, Total_loss: 2.011e+00, Loss_b: 7.589e-03, Loss_i: 6.499e-03, Loss_f: 6.022e-01, Time: 4.39\n", + "It: 940, Total_loss: 1.656e+00, Loss_b: 5.748e-03, Loss_i: 4.692e-03, Loss_f: 6.119e-01, Time: 4.34\n", + "It: 950, Total_loss: 1.524e+00, Loss_b: 5.275e-03, Loss_i: 4.060e-03, Loss_f: 5.907e-01, Time: 4.34\n", + "It: 960, Total_loss: 1.150e+00, Loss_b: 3.567e-03, Loss_i: 2.078e-03, Loss_f: 5.854e-01, Time: 4.36\n", + "It: 970, Total_loss: 1.136e+00, Loss_b: 3.517e-03, Loss_i: 2.214e-03, Loss_f: 5.624e-01, Time: 4.40\n", + "It: 980, Total_loss: 1.098e+00, Loss_b: 3.371e-03, Loss_i: 2.037e-03, Loss_f: 5.576e-01, Time: 4.31\n", + "It: 990, Total_loss: 1.058e+00, Loss_b: 3.218e-03, Loss_i: 1.868e-03, Loss_f: 5.492e-01, Time: 4.34\n", + "It: 1000, Total_loss: 1.048e+00, Loss_b: 3.260e-03, Loss_i: 1.985e-03, Loss_f: 5.237e-01, Time: 4.35\n", + "It: 1010, Total_loss: 9.346e+00, Loss_b: 3.993e-02, Loss_i: 4.750e-02, Loss_f: 6.030e-01, Time: 4.38\n", + "It: 1020, Total_loss: 5.139e+00, Loss_b: 2.211e-02, Loss_i: 2.336e-02, Loss_f: 5.921e-01, Time: 4.34\n", + "It: 1030, Total_loss: 2.316e+00, Loss_b: 8.761e-03, Loss_i: 8.724e-03, Loss_f: 5.671e-01, Time: 4.33\n", + "It: 1040, Total_loss: 1.314e+00, Loss_b: 4.416e-03, Loss_i: 3.425e-03, Loss_f: 5.302e-01, Time: 4.34\n", + "It: 1050, Total_loss: 1.101e+00, Loss_b: 3.623e-03, Loss_i: 2.288e-03, Loss_f: 5.101e-01, Time: 4.35\n", + "It: 1060, Total_loss: 9.831e-01, Loss_b: 2.902e-03, Loss_i: 1.671e-03, Loss_f: 5.259e-01, Time: 4.31\n", + "It: 1070, Total_loss: 9.554e-01, Loss_b: 2.894e-03, Loss_i: 1.657e-03, Loss_f: 5.003e-01, Time: 4.33\n", + "It: 1080, Total_loss: 9.228e-01, Loss_b: 2.796e-03, Loss_i: 1.600e-03, Loss_f: 4.832e-01, Time: 4.33\n", + "It: 1090, Total_loss: 9.017e-01, Loss_b: 2.708e-03, Loss_i: 1.540e-03, Loss_f: 4.769e-01, Time: 4.35\n", + "It: 1100, Total_loss: 9.491e-01, Loss_b: 2.927e-03, Loss_i: 1.858e-03, Loss_f: 4.705e-01, Time: 4.37\n", + "It: 1110, Total_loss: 6.445e+00, Loss_b: 2.654e-02, Loss_i: 3.269e-02, Loss_f: 5.224e-01, Time: 4.38\n", + "It: 1120, Total_loss: 5.876e+00, Loss_b: 2.427e-02, Loss_i: 2.891e-02, Loss_f: 5.584e-01, Time: 4.34\n", + "It: 1130, Total_loss: 1.434e+00, Loss_b: 5.268e-03, Loss_i: 4.353e-03, Loss_f: 4.723e-01, Time: 4.35\n" + ] + } + ], + "source": [ + "for epoch, lr in zip(epochs, learning_rate):\n", + " train(model, int(epoch), lr, batch_size=batch_size)\n", + "ms.save_checkpoint(model.dnn, f'model/{second_path}/model.ckpt')" + ] + }, + { + "cell_type": "markdown", + "id": "7032afa7-b415-4ef1-b49f-97a22e6ba634", + "metadata": {}, + "source": [ + "保存损失函数" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "9e56141a-8ec1-4d55-9f4e-cea19c209e8f", + "metadata": {}, + "outputs": [], + "source": [ + "np.save(f'Loss-Coe/{second_path}/loss_history_Adam_Pretrain', loss_history_Adam_Pretrain)\n", + "np.save(f'Loss-Coe/{second_path}/loss_b_history_Adam_Pretrain', loss_b_history_Adam_Pretrain)\n", + "np.save(f'Loss-Coe/{second_path}/loss_i_history_Adam_Pretrain', loss_i_history_Adam_Pretrain)\n", + "np.save(f'Loss-Coe/{second_path}/loss_f_history_Adam_Pretrain', loss_f_history_Adam_Pretrain)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.19" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/MindFlow/applications/research/NSFNets/README.md b/MindFlow/applications/research/NSFNets/README.md new file mode 100644 index 0000000000000000000000000000000000000000..d9f176ab488ccb3eaa68c73aa075169072c9ebf4 --- /dev/null +++ b/MindFlow/applications/research/NSFNets/README.md @@ -0,0 +1,76 @@ +# NSFNets: Physics-informed neural networks for the incompressible Navier-Stokes equations + +## Background 1 + +In the industrial application of AI-CFD, the scenarios for solving the NavierStokes equations and fusing multi-precision data are very extensive and have important economic and social significance. Solving ill-posed problems (such as missing some boundary conditions) or inversion problems is one of the key points and difficulties, and it is often costly, requiring the derivation of different formulas applicable to specific problems and the writing of new codes. How to solve the above problems with a set of unified codes at the same computational cost urgently needs in-depth research. Here, Jin Xiaowei, Li Hui and others used physical information neural networks (PINNs) to encode the control equations directly into deep neural networks through automatic differentiation to overcome some of the above-mentioned limitations of simulating incompressible laminar and turbulent flows. And developed Navier-Stokes flow networks (NSFnets, Navier-Stokes flow nets).. + +## Model framework + +The model framework is as shown in the following figure: + +![NSFNet](images/NSFNet.png) + +The figure shows the network for solving the Navier-Stokes equations in the pressure-velocity form. The automatic differentiation of the neural network is used to calculate the partial derivatives required in the equation. The loss function includes boundary condition loss, initial condition loss, and physical loss to satisfy the balance of the equation. + +## QuickStart + +The dataset can be directly generated by the provided code. + +The case provides two training methods + +- Run Option 1: Call `train.py` from command line + + ```python + # Call `train.py` from command line + python train.py --device_target GPU --device_id 0 --config_file_path ./config/NSFNet.yaml + + ``` + + `--config_path` indicates the path of the parameter file. Default path "./config/config.yaml". + + In the "./config/config.yaml" parameter file: + + 'case' represents the case name; + + 'device' representsthe type of computing platform used, with options of 'CPU', 'GPU', or 'Ascend', with a default value of 'GPU'; + + 'device_id' represents the index of NPU or GPU. Default 0; + + 'network_size' represents the size of network； + + 'learning_rate' represents the learning rate； + + 'epochs' represents number of training iterations； + +- Run Option 2: Run Jupyter Notebook + + You can use [chinese](./NSFNets_CN.ipynb) or [English](./NSFNets.ipynb) Jupyter Notebook to run the training and evaluation code line-by-line. + +## Case description + +The Jupyter Notebook named NSFNets runs the case of the three-dimensional Beltrami flow. In addition to providing the running code and data of this equation, this open source also provides the two-dimensional Kovasznay flow, which can be run directly by calling the `train_KF.py` script in the command line. + +## Performance + +| Parameters | GPU | NPU | +|:-------------------------:|:-----------------------:|:------------------:| +| hardware | NVIDIA 1080Ti(memory 11G) | Ascend(memory 32G) | +| MindSpore version | 2.2.14 | 2.2.14 | +| data_size | 91648 | 91648 | +| batch_size | 10000 | 10000 | +| epochs | 11w | 11w | +| optimizer | Adam | Adam | +| total loss ((MSE) | 2.6e-3 | 3.3e-4 | +| boundary loss ((MSE) | 5.9e-6 | 1.0e-5 | +| equation loss ((MSE) | 2.6e-6 | 6.3e-6 | +| regularization loss((MSE)| 1.8e-3 | 1.8e-3 | +| speed(s/step) | 0.43 | 0.068 | + +## Contributor + +gitee id: [chenchao2024](https://gitee.com/chenchao2024) +email: chenchao@isrc.iscas.ac.cn + +## Reference + +Jin x, Cai s, Li H, Karniadakis G. NSFnets (Navier-Stokes flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations[J]. Journal of Computational Physics, 2020, 426: 1. https://doi.org/10.1016/j.jcp.2020.109951 diff --git a/MindFlow/applications/research/NSFNets/README_CN.md b/MindFlow/applications/research/NSFNets/README_CN.md new file mode 100644 index 0000000000000000000000000000000000000000..5ca7f5b28c66000edf528bed64c8608078bfcb32 --- /dev/null +++ b/MindFlow/applications/research/NSFNets/README_CN.md @@ -0,0 +1,75 @@ +# NSFNets用于不可压缩 Navier-Stokes 方程求解的物理信息神经网络 + +## 背景介绍 + +在AI-CFD的工业应用方面，对NavierStokes方程的求解以及对多精度数据的融合的场景十分广泛，具有重要的经济和社会意义。求解不适定问题（例如部分边界条件缺失）或反演问题是其中的重点和难点之一，且往往代价高昂，需要推导适用于特定问题的不同公式、编写全新代码。如何用一套统一代码以相同计算代价解决上述问题亟需深入研究。在此，金晓威和李惠等使用物理信息神经网络（PINNs），通过自动微分将控制方程直接编码到深度神经网络中，以克服上述一些模拟不可压缩层流和湍流流动的限制。并开发了Navier-Stokes流动网络（NSFnets，Navier-Stokes flow nets）。 + +## 模型框架 + +模型框架如下图所示： + +![NSFNet](images/NSFNet.png) + +图中表示的是压力-速度形式的纳维-斯托克斯方程求解网络，利用神经网络的自动微分计算方程中所需要的偏导项，损失函数包括，边界条件损失，初始条件损失以及为了满足方程平衡的物理损失。 + +## 快速开始 + +数据集可以通过提供的代码直接生成。 + +案例提供两种训练方式 + +- 训练方式一：在命令行中调用`train.py`脚本 + + ```python + # 在命令行调用train.py进行训练示例 + python train.py --device_target GPU --device_id 0 --config_file_path ./config/NSFNet.yaml + + ``` + + `--config_path`表示配置文件的路径，默认值为 "./config/config.yaml"。 + + 在 "./config/config.yaml" 配置文件中： + 'case' 表示案例名称； + + 'device' 表示使用的计算平台类型，可以选择 'CPU'、'GPU' 或 'Ascend'，默认值 'GPU'； + + 'device_id' 表示后端平台端口号，默认值为 0； + + 'network_size' 表示网络大小； + + 'learning_rate' 表示学习率； + + 'epochs' 表示训练迭代次数； + +- 训练方式二：运行Jupyter Notebook + + 您可以使用[中文版](./NSFNets_CN.ipynb) 和[英文版](./NSFNets.ipynb) Jupyter Notebook逐行运行训练和验证代码。 + +## 案例说明 + +其中命名为NSFNets的Jupyter Notebook 运行的案例为三维Beltrami流动，本次开源除了提供该方程的运行代码和数据以外，还提供了二维Kovasznay流动，可以在命令行中调用`train_KF.py` 脚本即可直接运行。 + +## 性能 + +| 参数 | GPU | NPU | +|:-------------------:|:-----------------------:|:------------------:| +| 硬件 | NVIDIA 1080Ti(memory 11G) | Ascend(memory 32G) | +| MindSpore版本 | 2.2.14 | 2.2.14 | +| 数据大小 | 91648 | 91648 | +| batch大小 | 10000 | 10000 | +| 训练步数 | 11w | 11w | +| 优化器 | Adam | Adam | +| total loss 训练精度(MSE) | 2.6e-3 | 3.3e-3 | +| boundary loss 测试精度(MSE) | 5.9e-6 | 1.0e-5 | +| initial condition loss 训练精度(MSE) | 2.6e-6 | 6.3e-6 | +| equation loss 测试精度(MSE) | 1.7e-3 | 1.8e-3 | +| 性能(s/epoch) | 0.43 | 0.068 | + +## 贡献者 + +gitee id: [chenchao2024](https://gitee.com/chenchao2024) +email: chenchao@isrc.iscas.ac.cn + +## 参考文献 + +Jin x, Cai s, Li H, Karniadakis G. NSFnets (Navier-Stokes flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations[J]. Journal of Computational Physics, 2020, 426: 1. https://doi.org/10.1016/j.jcp.2020.109951 diff --git a/MindFlow/applications/research/NSFNets/config/NSFNet.yaml b/MindFlow/applications/research/NSFNets/config/NSFNet.yaml new file mode 100644 index 0000000000000000000000000000000000000000..469a881c59614c5bed83b55f457457e379aae76b --- /dev/null +++ b/MindFlow/applications/research/NSFNets/config/NSFNet.yaml @@ -0,0 +1,25 @@ +params: + model_name: 'NSFNet' + case: 'Three-dimensional-Beltrami-flow' + device: 'GPU' + device_id: '2' + load_params: False + load_params_test: True + second_path1: 'train' + network_size: [4, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 4] + learning_rate: [1.0e-03, 1.0e-04, 1.0e-05, 1.0e-06] + epochs: [5e3, 5e3, 5e4, 5e4] + batch_size: 10000 + xmin: -1 + xmax: 1 + ymin: -1 + ymax: 1 + zmin: -1 + zmax: 1 + tmin: 0. + tmax: 1. + n_x: 16 + n_y: 16 + n_z: 16 + n_t: 21 + re: 1 \ No newline at end of file diff --git a/MindFlow/applications/research/NSFNets/config/NSFNet_KF.yaml b/MindFlow/applications/research/NSFNets/config/NSFNet_KF.yaml new file mode 100644 index 0000000000000000000000000000000000000000..910fd6d0a5fbaffb73d56a9209ee32871b0cfa95 --- /dev/null +++ b/MindFlow/applications/research/NSFNets/config/NSFNet_KF.yaml @@ -0,0 +1,12 @@ +params: + model_name: 'NSFNet' + case: 'Kovasznay-Flow' + device: 'GPU' + device_id: '2' + load_params: False + load_params_test: True + second_path1: 'train' + network_size: [2, 50, 50, 50, 50, 3] + learning_rate: [1.0e-03, 1.0e-04, 1.0e-05, 1.0e-06] + epochs: [5e3, 5e3, 5e4, 5e4] + re: 40 \ No newline at end of file diff --git a/MindFlow/applications/research/NSFNets/images/NSFNet.png b/MindFlow/applications/research/NSFNets/images/NSFNet.png new file mode 100644 index 0000000000000000000000000000000000000000..ab661b771bc9d6c9bab7bf06bdf9f5a7557b7c2b Binary files /dev/null and b/MindFlow/applications/research/NSFNets/images/NSFNet.png differ diff --git a/MindFlow/applications/research/NSFNets/src/datasets.py b/MindFlow/applications/research/NSFNets/src/datasets.py new file mode 100644 index 0000000000000000000000000000000000000000..0f6d62a4e7d0d80c20d6f43a208dfb56aa33c5af --- /dev/null +++ b/MindFlow/applications/research/NSFNets/src/datasets.py @@ -0,0 +1,145 @@ +# Copyright 2023 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================ +"""NSFNet dataset""" +# pylint: disable=C0103 +import numpy as np + + +def ThreeBeltramiflow(x, y, z, t): + """ + ThreeBeltramiflow + """ + a = 1. + d = 1. + u = -a * (np.exp(a * x) * np.sin(a * y + d * z) + np.exp(a * z) * np.cos(a * x + d * y)) * np.exp(-d * d * t) + v = -a * (np.exp(a * y) * np.sin(a * z + d * x) + np.exp(a * x) * np.cos(a * y + d * z)) * np.exp(-d * d * t) + w = -a * (np.exp(a * z) * np.sin(a * x + d * y) + np.exp(a * y) * np.cos(a * z + d * x)) * np.exp(-d * d * t) + p = -0.5 * a * a * (np.exp(2. * a * x) + np.exp(2. * a * y) + np.exp(2. * a * z) + + 2. * np.sin(a * x + d * y) * np.cos(a * z + d * x) * np.exp(a * (y + z)) + + 2. * np.sin(a * y + d * z) * np.cos(a * x + d * y) * np.exp(a * (z + x)) + + 2. * np.sin(a * z + d * x) * np.cos(a * y + d * z) * np.exp(a * (x + y))) *\ + np.exp(-2. * d * d * t) + + return u, v, w, p + +def read_training_data(): + """prepare data""" + # prepare the training data + n_x = 16 + n_y = 16 + n_z = 16 + n_t = 21 + + + + + # Geometry + xmin = -1 + xmax = 1 + ymin = -1 + ymax = 1 + zmin = -1 + zmax = 1 + tmin = 0. + tmax = 1. + + # Boundary datapoints + t_ = np.arange(tmin, tmax + 1e-5, (tmax - tmin) / (n_t - 1.)).reshape([-1, 1]) + x_b_sum = np.arange(xmin, xmax + 1e-5, (xmax - xmin) / (n_x - 1.)) + y_b_sum = np.arange(ymin, ymax + 1e-5, (ymax - ymin) / (n_y - 1.)) + z_b_sum = np.arange(zmin, zmax + 1e-5, (zmax - zmin) / (n_z - 1.)) + + x_b_top, z_b_top = np.meshgrid(x_b_sum, z_b_sum) + y_b_top = ymax * np.ones([n_x * n_z, 1]) + x_b_top = x_b_top.reshape([-1, 1]) + z_b_top = z_b_top.reshape([-1, 1]) + + x_b_down, z_b_down = np.meshgrid(x_b_sum, z_b_sum) + y_b_down = ymin * np.ones([n_x * n_z, 1]) + x_b_down = x_b_down.reshape([-1, 1]) + z_b_down = z_b_down.reshape([-1, 1]) + + y_b_left, z_b_left = np.meshgrid(y_b_sum, z_b_sum) + x_b_left = xmin * np.ones([n_y * n_z, 1]) + y_b_left = y_b_left.reshape([-1, 1]) + z_b_left = z_b_left.reshape([-1, 1]) + + y_b_right, z_b_right = np.meshgrid(y_b_sum, z_b_sum) + x_b_right = xmax * np.ones([n_y * n_z, 1]) + y_b_right = y_b_right.reshape([-1, 1]) + z_b_right = z_b_right.reshape([-1, 1]) + + x_b_front, y_b_front = np.meshgrid(x_b_sum, y_b_sum) + z_b_front = zmax * np.ones([n_x * n_y, 1]) + x_b_front = x_b_front.reshape([-1, 1]) + y_b_front = x_b_front.reshape([-1, 1]) + + x_b_back, y_b_back = np.meshgrid(x_b_sum, y_b_sum) + z_b_back = zmin * np.ones([n_x * n_y, 1]) + x_b_back = x_b_back.reshape([-1, 1]) + y_b_back = x_b_back.reshape([-1, 1]) + + x_ = np.concatenate([x_b_top, x_b_down, x_b_left, x_b_right, x_b_front, x_b_back], axis=0).reshape([-1, 1]) + + y_ = np.concatenate([y_b_top, y_b_down, y_b_left, y_b_right, y_b_front, y_b_back], axis=0).reshape([-1, 1]) + + z_ = np.concatenate([z_b_top, z_b_down, z_b_left, z_b_right, z_b_front, z_b_back], axis=0).reshape([-1, 1]) + + T_ = np.tile(t_, (1, x_.shape[0])).T # N x T + X_ = np.tile(x_, (1, t_.shape[0])) # N x T + Y_ = np.tile(y_, (1, t_.shape[0])) # N x T + Z_ = np.tile(z_, (1, t_.shape[0])) # N x T + + t_b = T_.flatten()[:, None] # NT x 1 + x_b = X_.flatten()[:, None] # NT x 1 + y_b = Y_.flatten()[:, None] # NT x 1 + z_b = Z_.flatten()[:, None] # NT x 1 + + # data on the initial condition + x_ = np.arange(xmin, xmax + 1e-5, (xmax - xmin) / (n_x - 1.)) + y_ = np.arange(ymin, ymax + 1e-5, (ymax - ymin) / (n_y - 1.)) + z_ = np.arange(zmin, zmax + 1e-5, (zmax - zmin) / (n_z - 1.)) + xx, yy, zz = np.meshgrid(x_, y_, z_) + x_i = xx.flatten()[:, None] + y_i = yy.flatten()[:, None] + z_i = zz.flatten()[:, None] + t_i = np.ones([x_i.shape[0], 1]) * tmin + + t = np.concatenate([t_b, t_i], axis=0) + x = np.concatenate([x_b, x_i], axis=0) + y = np.concatenate([y_b, y_i], axis=0) + z = np.concatenate([z_b, z_i], axis=0) + + + u_b, v_b, w_b, _ = ThreeBeltramiflow(x_b, y_b, z_b, t_b) + u_i, v_i, w_i, _ = ThreeBeltramiflow(x_i, y_i, z_i, t_i) + + + + + X = np.concatenate([x, y, z, t], 1) + X_min = X.min(0) + X_max = X.max(0) + + # data on the velocity (inside the domain) + xx, yy, zz, tt = np.meshgrid(x_, y_, z_, t_) + + t_f = tt.flatten()[:, None] + x_f = xx.flatten()[:, None] + y_f = yy.flatten()[:, None] + z_f = zz.flatten()[:, None] + + + return x_b, y_b, z_b, t_b, x_i, y_i, z_i, t_i, u_b, v_b, w_b, u_i, v_i, w_i, x_f, y_f, z_f, t_f, X_min, X_max diff --git a/MindFlow/applications/research/NSFNets/src/datasets_kf.py b/MindFlow/applications/research/NSFNets/src/datasets_kf.py new file mode 100644 index 0000000000000000000000000000000000000000..a9c5686ba1f3ea987563aafb226b0b814fbb72e3 --- /dev/null +++ b/MindFlow/applications/research/NSFNets/src/datasets_kf.py @@ -0,0 +1,44 @@ +# Copyright 2023 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================ +"""NSFNet dataset""" +import numpy as np + + +def read_training_data(): + """prepare data""" + # prepare the training data + n_train = 2601 + n_bound = 4*101 + re = 40 + lam = 0.5 * re - np.sqrt(0.25 * (re ** 2) + 4 * (np.pi ** 2)) + x = np.linspace(-0.5, 1.0, n_bound + 1) + y = np.linspace(-0.5, 1.5, n_bound + 1) + yb1 = np.array([-0.5] * n_bound) + yb2 = np.array([1] * n_bound) + xb1 = np.array([-0.5] * n_bound) + xb2 = np.array([1.5] * n_bound) + + # stack the datapoints for generate the data + y_train1 = np.concatenate([y[1:n_bound + 1], y[0:n_bound], xb1, xb2], 0) + x_train1 = np.concatenate([yb1, yb2, x[0:n_bound], x[1:n_bound + 1]], 0) + xb_train = x_train1.reshape(x_train1.shape[0], 1) + yb_train = y_train1.reshape(y_train1.shape[0], 1) + + ub_train = 1 - np.exp(lam * xb_train) * np.cos(2 * np.pi * yb_train) + vb_train = lam / (2 * np.pi) * np.exp(lam * xb_train) * np.sin(2 * np.pi * yb_train) + + x_train = (np.random.rand(n_train, 1) - 1 / 3) * 3 / 2 # ([0 1)-1/3))*3/2 = [-1/3 2/3)*3/2 = [-0.5 1) + y_train = (np.random.rand(n_train, 1) - 1 / 4) * 2 # ([0 1) - 1/4) *2 = [-1/4 3/4)*2 = [-0.5 1.5) + return xb_train, yb_train, ub_train, vb_train, x_train, y_train diff --git a/MindFlow/applications/research/NSFNets/src/derivatives.py b/MindFlow/applications/research/NSFNets/src/derivatives.py new file mode 100644 index 0000000000000000000000000000000000000000..a62073d2fa468a4d4a46300ecd416105062ad52c --- /dev/null +++ b/MindFlow/applications/research/NSFNets/src/derivatives.py @@ -0,0 +1,504 @@ +# Copyright 2021 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================ +""" +derivative +""" +from inspect import isfunction +import numpy as np +import mindspore +import mindspore.numpy as mnp +from mindspore import Tensor, nn, ops, context +from mindspore.ops import constexpr +from mindspore import dtype as mstype + + +def check_mode(api_name): + if context.get_context("mode") == context.PYNATIVE_MODE: + raise RuntimeError("{} is only supported GRAPH_MODE now but got PYNATIVE_MODE".format(api_name)) + + +def check_type(param, param_name, param_type, exclude=None): + if (exclude and isinstance(param, exclude)) or not isinstance(param, param_type): + raise TypeError("The type of {} should be instance of {}, but got {}".format( + param_name, param_type, type(param))) + +def _transfer_tensor_to_tuple(inputs): + """ + If the input is a tensor, convert it to a tuple. If not, the output is unchanged. + """ + if isinstance(inputs, Tensor): + return (inputs,) + return inputs + + +class _GenerateMultiSens(nn.Cell): + """generate sens for multi-outputs""" + def construct(self, o, net_out, sens): + if len(net_out) == 1: + return sens + all_sens = () + for i in range(len(net_out)): + if i != o: + all_sens += (mnp.zeros(net_out[i].shape, mnp.float32),) + else: + all_sens += (sens,) + return all_sens + + +class _MergeOutput(nn.Cell): + """merge output""" + def construct(self, out_tmp, gout, iters): + for i in range(iters): + out_tmp[i] = out_tmp[i] + (gout[i],) + return out_tmp + + +@constexpr +def _generate_sens(batch_size, out_chanel, i): + sens = np.zeros((batch_size, out_chanel), np.float32) + sens[:, i] = 1 + return Tensor(sens) + + +@constexpr +def _generate_indices(j): + return Tensor([j], mindspore.int32) + + +@constexpr +def _check_type(net_in, net_out, input_idx=None, output_idx=None): + """check type of input""" + if net_in is not None: + raise TypeError("The Type of network input should be Tensor but got {}".format(type(net_in))) + if input_idx is not None and (not isinstance(input_idx, int) or isinstance(input_idx, bool)): + raise TypeError("The Type of column index of input should be int but got {}".format(type(input_idx))) + if output_idx is not None and (not isinstance(output_idx, int) or isinstance(output_idx, bool)): + raise TypeError("The Type of column index of output should be int but got {}".format(type(output_idx))) + if net_out is not None: + raise TypeError("The Type of network output should be Tensor but got {}".format(type(net_out))) + + +@constexpr +def _check_dimension(in_shape, out_shape, in_idx, out_idx): + """check dimension of input""" + if len(in_shape) != 2: + raise ValueError("The dimension of network input should be 2, but got {}".format(len(in_shape))) + if len(out_shape) != 2: + raise ValueError("The dimension of network output should be 2, but got {}".format(len(out_shape))) + if in_idx is not None and out_idx is not None: + if in_idx >= in_shape[1]: + raise ValueError("input index should be in range (0, {}), but got {}".format(in_shape[1], in_idx)) + if out_idx >= out_shape[1]: + raise ValueError("output index should be in range (0, {}), but got {}".format(out_shape[1], out_idx)) + + +class Grad(nn.Cell): + """ + Computes and returns the gradients of the specified column of outputs with respect to the specified column of + inputs. + + Args: + model (Cell): a function or network that takes Tensor inputs. + argnum (int): specifies which input the output takes the first derivative of. Default: ``0``. + + Inputs: + - **x** (list) - The input is variable-length argument. The first input is a 2D network inputs (Tensor), + the last three inputs are column index of input (int), column index of output (int) and output of + network (Tensor). + + Outputs: + Tensor. The gradients of the specified column of outputs with respect to the specified column of + inputs. + + Raises: + TypeError: If the type of `argnum` is not int. + + Supported Platforms: + ``Ascend`` + + Examples: + >>> import numpy as np + >>> from mindspore import nn, Tensor + >>> from mindelec.operators import Grad + ... + >>> class Net(nn.Cell): + ... def __init__(self): + ... super(Net, self).__init__() + ... def construct(self, x): + ... return x * x + ... + >>> x = Tensor(np.array([[1.0, -2.0], [-3.0, 4.0]]).astype(np.float32)) + >>> net = Net() + >>> out = net(x) + >>> grad = Grad(net) + >>> print(grad(x, 0, 0, out).asnumpy()) + [[ 2.] + [-6.]] + """ + def __init__(self, model, argnum=0): + super(Grad, self).__init__() + check_mode("Grad") + if not isinstance(model, nn.Cell) and not isfunction(model): + raise TypeError("The type of model should be a function or network, but got {}".format(type(model))) + self.model = model + if isinstance(argnum, bool) or not isinstance(argnum, int): + raise TypeError("The type of argnum should be int, but get {}".format(type(argnum))) + self.argnum = argnum + self.grad = ops.GradOperation(get_all=True, sens_param=True) + self.gather = ops.Gather() + self.cast = ops.Cast() + self.dtype = ops.DType() + + def construct(self, *x): + """ + construct + """ + x = _transfer_tensor_to_tuple(x) + input_idx, output_idx, net_out = x[-3], x[-2], x[-1] + net_in = x[:-3] + _check_type(net_in[0], net_out, input_idx, output_idx) + if net_out is None: + net_out = self.model(*net_in) + net_out = _transfer_tensor_to_tuple(net_out)[0] + _check_dimension(net_in[self.argnum].shape, net_out.shape, input_idx, output_idx) + batch_size, out_chanel = net_out.shape + sens = _generate_sens(batch_size, out_chanel, output_idx) + gradient_function = self.grad(self.model) + sens = self.cast(sens, self.dtype(net_out)) + gradient = gradient_function(*net_in, sens) + if input_idx is None: + output = gradient[self.argnum] + else: + out_indices = _generate_indices(input_idx) + output = self.gather(gradient[self.argnum], out_indices, 1) + return output + + +class SecondOrderGrad(nn.Cell): + """ + Computes and returns the second order gradients of the specified column of outputs with respect to the specified + column of inputs. + + Args: + model (Cell): a function or network that takes a single Tensor input and returns a single Tensor. + input_idx1 (int): specifies the column index of input to take the first derivative, + takes values in [0, model input size - 1]. + input_idx2 (int): specifies the column index of input to take the second derivative, + takes values in [0, model input size - 1]. + output_idx (int): specifies the column index of output, takes values in [0, model output size - 1]. + + Inputs: + - **input** - The input of given function or network `model`. + + Outputs: + Tensor. + + Raises: + TypeError: If the type of `input_idx1`, `input_idx2` or `output_idx` is not int. + + Supported Platforms: + ``Ascend`` + + Examples: + >>> import numpy as np + >>> from mindspore import nn, Tensor + >>> from mindelec.operators import SecondOrderGrad + >>> class Net(nn.Cell): + ... def __init__(self): + ... super(Net, self).__init__() + ... + ... def construct(self, x): + ... return x * x * x + >>> x = Tensor(np.array([[1.0, -2.0], [-3.0, 4.0]]).astype(np.float32)) + >>> net = Net() + >>> out = net(x) + >>> grad = SecondOrderGrad(net, 0, 0, 0) + >>> print(grad(x).asnumpy()) + [[ 6.] + [-18.]] + """ + def __init__(self, model, input_idx1, input_idx2, output_idx): + super(SecondOrderGrad, self).__init__() + check_mode("SecondOrderGrad") + if not isinstance(model, nn.Cell) and not isfunction(model): + raise TypeError("The type of model should be a function or network, but got {}".format(type(model))) + if isinstance(input_idx1, bool) or not isinstance(input_idx1, int): + raise TypeError("The type of input_idx1 should be int, but got {}".format(type(input_idx1))) + if isinstance(input_idx2, bool) or not isinstance(input_idx2, int): + raise TypeError("The type of input_idx1 should be int, but got {}".format(type(input_idx2))) + if isinstance(output_idx, bool) or not isinstance(output_idx, int): + raise TypeError("The type of input_idx1 should be int, but got {}".format(type(output_idx))) + self.jac1 = _FirstOrderGrad(model, input_idx=input_idx1, output_idx=output_idx) + self.jac2 = _FirstOrderGrad(self.jac1, input_idx=input_idx2, output_idx=0) + + def construct(self, x): + hes = self.jac2(x) + return hes + +class ThirdOrderGrad(nn.Cell): + """ + Computes and returns the Third order gradients of the specified column of outputs with respect to the specified + column of inputs. + """ + def __init__(self, model, input_idx1, input_idx2, output_idx): + super(ThirdOrderGrad, self).__init__() + check_mode("SecondOrderGrad") + if not isinstance(model, nn.Cell) and not isfunction(model): + raise TypeError("The type of model should be a function or network, but got {}".format(type(model))) + if isinstance(input_idx1, bool) or not isinstance(input_idx1, int): + raise TypeError("The type of input_idx1 should be int, but got {}".format(type(input_idx1))) + if isinstance(input_idx2, bool) or not isinstance(input_idx2, int): + raise TypeError("The type of input_idx1 should be int, but got {}".format(type(input_idx2))) + if isinstance(output_idx, bool) or not isinstance(output_idx, int): + raise TypeError("The type of input_idx1 should be int, but got {}".format(type(output_idx))) + self.jac1 = _FirstOrderGrad(model, input_idx=input_idx1, output_idx=output_idx) + self.jac2 = _FirstOrderGrad(self.jac1, input_idx=input_idx2, output_idx=0) + self.jac3 = _FirstOrderGrad(self.jac2, input_idx=input_idx2, output_idx=0) + + def construct(self, x): + hes = self.jac3(x) + return hes + +class ForthOrderGrad(nn.Cell): + """ + Computes and returns the Third order gradients of the specified column of outputs with respect to the specified + column of inputs. + """ + def __init__(self, model, input_idx1, input_idx2, output_idx): + super(ForthOrderGrad, self).__init__() + check_mode("SecondOrderGrad") + if not isinstance(model, nn.Cell) and not isfunction(model): + raise TypeError("The type of model should be a function or network, but got {}".format(type(model))) + if isinstance(input_idx1, bool) or not isinstance(input_idx1, int): + raise TypeError("The type of input_idx1 should be int, but got {}".format(type(input_idx1))) + if isinstance(input_idx2, bool) or not isinstance(input_idx2, int): + raise TypeError("The type of input_idx1 should be int, but got {}".format(type(input_idx2))) + if isinstance(output_idx, bool) or not isinstance(output_idx, int): + raise TypeError("The type of input_idx1 should be int, but got {}".format(type(output_idx))) + self.jac1 = _FirstOrderGrad(model, input_idx=input_idx1, output_idx=output_idx) + self.jac2 = _FirstOrderGrad(self.jac1, input_idx=input_idx2, output_idx=0) + self.jac3 = _FirstOrderGrad(self.jac2, input_idx=input_idx2, output_idx=0) + self.jac4 = _FirstOrderGrad(self.jac3, input_idx=input_idx2, output_idx=0) + + def construct(self, x): + hes = self.jac4(x) + return hes + + +class Jacobian(nn.Cell): + r""" + Computes the Jacobian of a given function or network. + + Note: + The output of the given function or network should be a single Tensor. + + Args: + net (Union[function, Cell]): a function or network that takes Tensor inputs. + arg_nums (int): specifies which input the output takes the first derivative of. + out_idx (int): specifies which output to take the first derivative of. + + Inputs: + - **x** (Tensor) - The inputs of the function or network `net`. + + Outputs: + Tensor or tuple of Tensors. If `arg_nums` is int, output will be a Tensor whose shape is the shape of + specified output * the shape of specified input. If `arg_nums` is None, output will be a tuple of Tensors + where output[i] will contain the Jacobian of the specified output and ith input and will have as size the + concatenation of the sizes of the corresponding output and the corresponding input. + + Raises: + TypeError: if the type of `arg_nums` or `out_idx` is not int. + + Supported Platforms: + ``Ascend`` + + Examples: + >>> import numpy as np + >>> from mindelec.operators import Jacobian + >>> from mindspore import Tensor + >>> def func(x, y): + >>> return (x * x * x * y + 3 * y * y * x).sum() + ... + >>> a = Tensor(np.array([[1, 3], [5, 9], [8, 2]], np.float32)) + >>> b = Tensor(np.array([[4, 6], [7, 2], [2, 1]], np.float32)) + >>> jac = Jacobian(func, 0, 0) + >>> output = jac(a, b) + >>> print(output.shape) + (3, 2) + """ + def __init__(self, net, argnums=0, out_idx=0): + super(Jacobian, self).__init__() + if not (isinstance(argnums, int) or argnums is None) or not isinstance(out_idx, int): + raise TypeError("The type of argnums should be int or None and out_idx should be int.") + self.net = net + self.argnums = argnums + self.out_idx = out_idx + self.grad_op = ops.GradOperation(get_all=True, sens_param=True) + self.eye = ops.Eye() + self.concat = ops.Concat() + self.reshape = ops.Reshape() + self.tuple_len = ops.tuple_len + self._merge_output = _MergeOutput() + self._generate_multi_sens = _GenerateMultiSens() + + def construct(self, *x): + """ + forward + + Args: + inputs (tuple): input tensor. + """ + net_out = _transfer_tensor_to_tuple(self.net(*x)) + net_out_target = net_out[self.out_idx] + grad_fn = self.grad_op(self.net) + input_len = self.tuple_len(x) + + identity_matrix = self.eye(net_out_target.size, net_out_target.size, mstype.float32) + identity_matrix = ops.Split(0, net_out_target.size)(identity_matrix) + if self.argnums is None: + out_tmp = [()] * input_len + for line in identity_matrix: + sens = self.reshape(line, net_out_target.shape) + grad_wrt_output = self._generate_multi_sens(self.out_idx, net_out, sens) + grad = grad_fn(*x, grad_wrt_output) + out_tmp = self._merge_output(out_tmp, grad, input_len) + output = () + for i in range(input_len): + out_tmp[i] = self.concat(out_tmp[i]) + output = output + (self.reshape(out_tmp[i], net_out_target.shape + x[i].shape),) + return output + output = () + for line in identity_matrix: + sens = self.reshape(line, net_out_target.shape) + grad_wrt_output = self._generate_multi_sens(self.out_idx, net_out, sens) + grad = grad_fn(*x, grad_wrt_output) + output = output + (grad[self.argnums],) + output = self.concat(output) + return self.reshape(output, net_out_target.shape + x[self.argnums].shape) + + +class Hessian(nn.Cell): + r""" + Computes the Hessian of a given function or network. + + Note: + The output of the given function or network should be a single Tensor. + + Args: + net (Union[function, Cell]): a function or network that takes Tensor inputs and returns a single Tensor. + diff1_idx (int): specifies which input the output takes the first derivative of. + diff2_idx (int): specifies which input the output takes the second derivative of. + + Inputs: + - **x** (Tensor) - The inputs of the function or network `net`. + + Outputs: + Tensor, the shape is the shape output * shape of specified input * the shape of specified input. + + Raises: + TypeError: if the type of `diff1_idx` or `diff2_idx` is not int. + + Supported Platforms: + ``Ascend`` + + Examples: + >>> import numpy as np + >>> from mindelec.operators import Hessian + >>> from mindspore import Tensor + >>> def func(x, y): + >>> return (x * x * x * y + 3 * y * y * x).sum() + >>> a = Tensor(np.array([[1, 3], [5, 9], [8, 2]], np.float32)) + >>> b = Tensor(np.array([[4, 6], [7, 2], [2, 1]], np.float32)) + >>> hes = Hessian(func, 0, 0) + >>> output = hes(a, b) + >>> print(output.shape) + (3, 2, 3, 2) + """ + def __init__(self, net, diff1_idx, diff2_idx, out_idx=0): + super(Hessian, self).__init__() + if not isinstance(diff1_idx, int) or not (isinstance(diff2_idx, int) or diff2_idx is None): + raise TypeError("The type of diff1 should be int and diff2 should be int or None.") + self.jac1 = Jacobian(net, argnums=None, out_idx=out_idx) + self.jac2 = Jacobian(self.jac1, argnums=diff2_idx, out_idx=diff1_idx) + + def construct(self, *x): + return self.jac2(*x) + + +def jacobian(func, inputs): + r""" + Function that computes the Jacobian of a given function or network. + + Parameters: + func: a function or network that takes Tensor inputs. + inputs: The inputs of the function or network `net`. + """ + inputs = _transfer_tensor_to_tuple(inputs) + func_out = _transfer_tensor_to_tuple(func(*inputs)) + output = () + for i in range(len(func_out)): + jac = Jacobian(func, argnums=None, out_idx=i) + output = output + _transfer_tensor_to_tuple(jac(*inputs)) + return output + + +def hessian(func, inputs): + r""" + Function that computes the Hessian of a given function or network. + + Parameters: + func: a function or network that takes Tensor inputs. + inputs: The inputs of the function or network `net`. + """ + inputs = _transfer_tensor_to_tuple(inputs) + func_out = _transfer_tensor_to_tuple(func(*inputs)) + output = () + for i in range(len(func_out)): + out_tmp = () + for j in range(len(inputs)): + hes = Hessian(func, diff1_idx=j, diff2_idx=None, out_idx=i) + out_tmp = out_tmp + _transfer_tensor_to_tuple(hes(*inputs)) + output = output + out_tmp + return output + + +class _FirstOrderGrad(nn.Cell): + """compute first-order derivative""" + def __init__(self, model, argnums=0, input_idx=None, output_idx=1): + super(_FirstOrderGrad, self).__init__() + self.model = model + self.argnums = argnums + self.input_idx = input_idx + self.output_idx = output_idx + self.grad = ops.GradOperation(get_all=True, sens_param=True) + self.gather = ops.Gather() + self.cast = ops.Cast() + self.dtype = ops.DType() + + def construct(self, *x): + """Defines the computation to be performed""" + x = _transfer_tensor_to_tuple(x) + _check_type(x[self.argnums], None) + net_out = self.model(*x) + net_out = _transfer_tensor_to_tuple(net_out)[0] + _check_dimension(x[self.argnums].shape, net_out.shape, self.input_idx, self.output_idx) + batch_size, out_chanel = net_out.shape + sens = _generate_sens(batch_size, out_chanel, self.output_idx) + gradient_function = self.grad(self.model) + sens = self.cast(sens, self.dtype(net_out)) + gradient = gradient_function(*x, sens) + outout_indices = _generate_indices(self.input_idx) + output = self.gather(gradient[self.argnums], outout_indices, 1) + return output diff --git a/MindFlow/applications/research/NSFNets/src/network.py b/MindFlow/applications/research/NSFNets/src/network.py new file mode 100644 index 0000000000000000000000000000000000000000..8e1c375d599e36029445c830301ea2c18cd32cf9 --- /dev/null +++ b/MindFlow/applications/research/NSFNets/src/network.py @@ -0,0 +1,213 @@ +# Copyright 2023 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================ +"""Network definitions""" +# pylint: disable=C0103 +import mindspore as ms +from mindspore import nn, ops +from mindspore.common.api import jit +from mindspore.amp import DynamicLossScaler +import numpy as np +from src.derivatives import SecondOrderGrad, Grad + + +# the deep neural network +class neural_net(nn.Cell): + """ + neural_net + """ + def __init__(self, layers, msfloat_type, lb, ub): + super(neural_net, self).__init__() + + # normalize the scope + self.lb = lb + self.ub = ub + + # network parameters + self.depth = len(layers) - 1 + + # set up activation + self.activation = nn.Tanh() + + layer_list = [] + for i in range(self.depth - 1): + layer_list.append(nn.Dense(layers[i], layers[i+1]).to_float(msfloat_type)) + layer_list.append(self.activation.to_float(msfloat_type)) + + layer_list.append(nn.Dense(layers[-2], layers[-1]).to_float(msfloat_type)) + + # deploy layers + self.layers = nn.SequentialCell(layer_list).to_float(msfloat_type) + + @jit + def construct(self, x): + H = 2.0 * (x - self.lb) / (self.ub - self.lb) - 1.0 + out = self.layers(H) + return out + +class VPNSFNets: + """ + VPNSFNets + """ + # Initialize the class + def __init__(self, xb, yb, zb, tb, xi, yi, zi, ti, ub, vb, wb, ui, vi, wi, x_f, y_f, z_f, t_f, layers, Re, \ + Xmin, Xmax, use_npu, msfloat_type, npfloat_type, load_params, second_path): + + self.use_npu = use_npu + + self.msfloat_type = msfloat_type + self.npfloat_type = npfloat_type + + self.Xmin = ms.Tensor(np.array(Xmin, self.npfloat_type)) + self.Xmax = ms.Tensor(np.array(Xmax, self.npfloat_type)) + + # The size of network + self.layers = layers + self.Re = Re + + # Training data + self.x_f = ms.Tensor(np.array(x_f, self.npfloat_type)) + self.y_f = ms.Tensor(np.array(y_f, self.npfloat_type)) + self.z_f = ms.Tensor(np.array(z_f, self.npfloat_type)) + self.t_f = ms.Tensor(np.array(t_f, self.npfloat_type)) + + self.xb = ms.Tensor(np.array(xb, self.npfloat_type)) + self.yb = ms.Tensor(np.array(yb, self.npfloat_type)) + self.zb = ms.Tensor(np.array(zb, self.npfloat_type)) + self.tb = ms.Tensor(np.array(tb, self.npfloat_type)) + + self.xi = ms.Tensor(np.array(xi, self.npfloat_type)) + self.yi = ms.Tensor(np.array(yi, self.npfloat_type)) + self.zi = ms.Tensor(np.array(zi, self.npfloat_type)) + self.ti = ms.Tensor(np.array(ti, self.npfloat_type)) + + self.ub = ms.Tensor(np.array(ub, self.npfloat_type)) + self.vb = ms.Tensor(np.array(vb, self.npfloat_type)) + self.wb = ms.Tensor(np.array(wb, self.npfloat_type)) + + self.ui = ms.Tensor(np.array(ui, self.npfloat_type)) + self.vi = ms.Tensor(np.array(vi, self.npfloat_type)) + self.wi = ms.Tensor(np.array(wi, self.npfloat_type)) + + # Initialize NNs---deep neural networks + self.dnn = neural_net(layers, self.msfloat_type, self.Xmin, self.Xmax) + if load_params: + params_dict = ms.load_checkpoint(f'model/{second_path}/model.ckpt') + ms.load_param_into_net(self.dnn, params_dict) + + self.grad = Grad(self.dnn) + self.hessian_u_xx = SecondOrderGrad(self.dnn, 0, 0, output_idx=0) + self.hessian_u_yy = SecondOrderGrad(self.dnn, 1, 1, output_idx=0) + self.hessian_u_zz = SecondOrderGrad(self.dnn, 2, 2, output_idx=0) + + self.hessian_v_xx = SecondOrderGrad(self.dnn, 0, 0, output_idx=1) + self.hessian_v_yy = SecondOrderGrad(self.dnn, 1, 1, output_idx=1) + self.hessian_v_zz = SecondOrderGrad(self.dnn, 2, 2, output_idx=1) + + self.hessian_w_xx = SecondOrderGrad(self.dnn, 0, 0, output_idx=2) + self.hessian_w_yy = SecondOrderGrad(self.dnn, 1, 1, output_idx=2) + self.hessian_w_zz = SecondOrderGrad(self.dnn, 2, 2, output_idx=2) + + self.loss_scaler = DynamicLossScaler(1024, 2, 100) + + def net_u(self, x, y, z, t): + H = ms.ops.concat([x, y, z, t], 1) + Y = self.dnn(H) + return Y + + def net_f(self, x_f, y_f, z_f, t_f): + """ The minspore autograd version of calculating residual """ + U = self.net_u(x_f, y_f, z_f, t_f) + u = U[:, 0:1] + v = U[:, 1:2] + w = U[:, 2:3] + p = U[:, 3:4] + + data = ms.ops.concat([x_f, y_f, z_f, t_f], 1) + + u_x = self.grad(data, 0, 0, U) + u_y = self.grad(data, 1, 0, U) + u_z = self.grad(data, 2, 0, U) + u_t = self.grad(data, 3, 0, U) + u_xx = self.hessian_u_xx(data) + u_yy = self.hessian_u_yy(data) + u_zz = self.hessian_u_zz(data) + + v_x = self.grad(data, 0, 1, U) + v_y = self.grad(data, 1, 1, U) + v_z = self.grad(data, 2, 1, U) + v_t = self.grad(data, 3, 1, U) + v_xx = self.hessian_v_xx(data) + v_yy = self.hessian_v_yy(data) + v_zz = self.hessian_v_zz(data) + + w_x = self.grad(data, 0, 2, U) + w_y = self.grad(data, 1, 2, U) + w_z = self.grad(data, 2, 2, U) + w_t = self.grad(data, 3, 2, U) + w_xx = self.hessian_w_xx(data) + w_yy = self.hessian_w_yy(data) + w_zz = self.hessian_w_zz(data) + + p_x = self.grad(data, 0, 3, U) + p_y = self.grad(data, 1, 3, U) + p_z = self.grad(data, 2, 3, U) + + f_u = u_t + (u * u_x + v * u_y + w * u_z) + p_x - 1. / self.Re * (u_xx + u_yy + u_zz) + f_v = v_t + (u * v_x + v * v_y + w * v_z) + p_y - 1. / self.Re * (v_xx + v_yy + v_zz) + f_w = w_t + (u * w_x + v * w_y + w * w_z) + p_z - 1. / self.Re * (w_xx + w_yy + w_zz) + + f_e = u_x + v_y + w_z + + return u, v, w, p, f_u, f_v, f_w, f_e + + def loss_fn(self, xb, yb, zb, tb, xi, yi, zi, ti, x_f, y_f, z_f, t_f, ub, vb, wb, ui, vi, wi): + """ + loss_fn + """ + Ub = self.net_u(xb, yb, zb, tb) + ub_pred = Ub[:, 0:1] + vb_pred = Ub[:, 1:2] + wb_pred = Ub[:, 2:3] + + Ui = self.net_u(xi, yi, zi, ti) + ui_pred = Ui[:, 0:1] + vi_pred = Ui[:, 1:2] + wi_pred = Ui[:, 2:3] + + _, _, _, _, f_u_pred, f_v_pred, f_w_pred, f_e_pred = self.net_f(x_f, y_f, z_f, t_f) + + loss_ub = ops.reduce_mean(ops.square(ub_pred - ub)) + loss_vb = ops.reduce_mean(ops.square(vb_pred - vb)) + loss_wb = ops.reduce_mean(ops.square(wb_pred - wb)) + + loss_ui = ops.reduce_mean(ops.square(ui_pred - ui)) + loss_vi = ops.reduce_mean(ops.square(vi_pred - vi)) + loss_wi = ops.reduce_mean(ops.square(wi_pred - wi)) + + alpha = ms.Tensor(100., dtype=self.msfloat_type) + + loss_f_u = ops.reduce_mean(ops.square(f_u_pred - ms.ops.zeros_like(f_u_pred))) + loss_f_v = ops.reduce_mean(ops.square(f_v_pred - ms.ops.zeros_like(f_v_pred))) + loss_f_w = ops.reduce_mean(ops.square(f_w_pred - ms.ops.zeros_like(f_w_pred))) + loss_f_e = ops.reduce_mean(ops.square(f_e_pred - ms.ops.zeros_like(f_e_pred))) + + loss_b = loss_ub + loss_vb + loss_wb + loss_i = loss_ui + loss_vi + loss_wi + loss_f = loss_f_u + loss_f_v + loss_f_w + loss_f_e + + loss = alpha * loss_b + alpha * loss_i + loss_f + if self.use_npu: + loss = self.loss_scaler.scale(loss) + return loss, loss_b, loss_i, loss_f diff --git a/MindFlow/applications/research/NSFNets/src/network_kf.py b/MindFlow/applications/research/NSFNets/src/network_kf.py new file mode 100644 index 0000000000000000000000000000000000000000..8a42b0e875a3ce97f40ed2c5fb19b4a70d5294d0 --- /dev/null +++ b/MindFlow/applications/research/NSFNets/src/network_kf.py @@ -0,0 +1,169 @@ +# Copyright 2023 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================ +"""Network definitions""" +# pylint: disable=C0103 +import mindspore as ms +from mindspore import nn, ops +from mindspore.common.api import jit +from mindspore.amp import DynamicLossScaler +import numpy as np +from src.derivatives import SecondOrderGrad, Grad + + +# the deep neural network +class neural_net(nn.Cell): + """ + neural_net + """ + def __init__(self, layers, msfloat_type, lb, ub): + super(neural_net, self).__init__() + + # normalize the scope + self.lb = lb + self.ub = ub + + # network parameters + self.depth = len(layers) - 1 + + # set up activation + self.activation = nn.Tanh() + + layer_list = [] + for i in range(self.depth - 1): + layer_list.append(nn.Dense(layers[i], layers[i+1]).to_float(msfloat_type)) + layer_list.append(self.activation.to_float(msfloat_type)) + + layer_list.append(nn.Dense(layers[-2], layers[-1]).to_float(msfloat_type)) + + # deploy layers + self.layers = nn.SequentialCell(layer_list).to_float(msfloat_type) + + @jit + def construct(self, x): + H = 2.0 * (x - self.lb) / (self.ub - self.lb) - 1.0 + out = self.layers(H) + return out + +class VPNSFNets: + """ + VPNSFNets + """ + # Initialize the class + def __init__(self, xb, yb, ub, vb, x_f, y_f, layers, use_npu, + msfloat_type, npfloat_type, load_params, second_path): + + self.use_npu = use_npu + + self.msfloat_type = msfloat_type + self.npfloat_type = npfloat_type + + self.Xb_l = ops.concat([ms.Tensor(np.array(xb, self.npfloat_type)), + ms.Tensor(np.array(yb, self.npfloat_type))], 1) + self.X_l = ops.concat([ms.Tensor(np.array(x_f, self.npfloat_type)), + ms.Tensor(np.array(y_f, self.npfloat_type))], 1) + + self.Xmin = ms.Tensor(np.array(self.Xb_l.min(0), self.npfloat_type)) + self.Xmax = ms.Tensor(np.array(self.Xb_l.max(0), self.npfloat_type)) + + + self.layers = layers + + self.x_f = ms.Tensor(np.array(x_f, self.npfloat_type)) + self.y_f = ms.Tensor(np.array(y_f, self.npfloat_type)) + + self.xb = ms.Tensor(np.array(xb, self.npfloat_type)) + self.yb = ms.Tensor(np.array(yb, self.npfloat_type)) + self.ub = ms.Tensor(np.array(ub, self.npfloat_type)) + self.vb = ms.Tensor(np.array(vb, self.npfloat_type)) + + # Initialize NNs---deep neural networks + self.dnn = neural_net(layers, self.msfloat_type, self.Xmin, self.Xmax) + if load_params: + params_dict = ms.load_checkpoint(f'model/{second_path}/model.ckpt') + ms.load_param_into_net(self.dnn, params_dict) + + # The function of Auto-differentiation + self.grad = Grad(self.dnn) + self.hessian_u_xx = SecondOrderGrad(self.dnn, 0, 0, output_idx=0) + self.hessian_u_yy = SecondOrderGrad(self.dnn, 1, 1, output_idx=0) + + self.hessian_v_xx = SecondOrderGrad(self.dnn, 0, 0, output_idx=1) + self.hessian_v_yy = SecondOrderGrad(self.dnn, 1, 1, output_idx=1) + + self.loss_scaler = DynamicLossScaler(1024, 2, 100) + + def net_u(self, x, y): + H = ms.ops.concat([x, y], 1) + Y = self.dnn(H) + return Y + + def net_f(self, x_f, y_f): + """ + net_f + """ + U = self.net_u(x_f, y_f) + u = U[:, 0:1] + v = U[:, 1:2] + p = U[:, 2:3] + + data = ms.ops.concat([x_f, y_f], 1) + + u_x = self.grad(data, 0, 0, U) + u_y = self.grad(data, 1, 0, U) + u_xx = self.hessian_u_xx(data) + u_yy = self.hessian_u_yy(data) + + v_x = self.grad(data, 0, 1, U) + v_y = self.grad(data, 1, 1, U) + v_xx = self.hessian_v_xx(data) + v_yy = self.hessian_v_yy(data) + + p_x = self.grad(data, 0, 2, U) + p_y = self.grad(data, 1, 2, U) + + f_u = (u * u_x + v * u_y) + p_x - (1.0/40) * (u_xx + u_yy) + f_v = (u * v_x + v * v_y) + p_y - (1.0/40) * (v_xx + v_yy) + f_e = u_x + v_y + + return u, v, p, f_u, f_v, f_e + + + def loss_fn(self, xb, yb, x_f, y_f, ub, vb): + """ + loss_fn + """ + Ub = self.net_u(xb, yb) + ub_pred = Ub[:, 0:1] + vb_pred = Ub[:, 1:2] + + + _, _, _, f_u_pred, f_v_pred, f_e_pred = self.net_f(x_f, y_f) + + loss_ub = ops.reduce_mean(ops.square(ub_pred - ub)) + loss_vb = ops.reduce_mean(ops.square(vb_pred - vb)) + + alpha = ms.Tensor(1., dtype=self.msfloat_type) + + loss_f_u = ops.reduce_mean(ops.square(f_u_pred - ms.ops.zeros_like(f_u_pred))) + loss_f_v = ops.reduce_mean(ops.square(f_v_pred - ms.ops.zeros_like(f_v_pred))) + loss_f_e = ops.reduce_mean(ops.square(f_e_pred - ms.ops.zeros_like(f_e_pred))) + + loss_b = loss_ub + loss_vb + loss_f = loss_f_u + loss_f_v + loss_f_e + + loss = alpha * loss_b + loss_f + if self.use_npu: + loss = self.loss_scaler.scale(loss) + return loss, loss_b, loss_f diff --git a/MindFlow/applications/research/NSFNets/test.py b/MindFlow/applications/research/NSFNets/test.py new file mode 100644 index 0000000000000000000000000000000000000000..4e4efd8bdffc9e20e93b63720edf3978d281968e --- /dev/null +++ b/MindFlow/applications/research/NSFNets/test.py @@ -0,0 +1,165 @@ +# Copyright 2023 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================ +"""ICNet train""" +# pylint: disable=C0103 +# pylint: disable=E1121 +import os +import argparse + +import numpy as np +import scipy.io +import mindspore as ms +from mindspore import set_seed, context +from mindflow.utils import load_yaml_config +from src.network import VPNSFNets +from src.datasets import read_training_data, ThreeBeltramiflow + +# Adam loss history +loss_history_adam_pretrain = np.empty([0]) +loss_b_history_adam_pretrain = np.empty([0]) +loss_i_history_adam_pretrain = np.empty([0]) +loss_f_history_adam_pretrain = np.empty([0]) +np.random.seed(123456) +set_seed(123456) + +if __name__ == "__main__": + parser = argparse.ArgumentParser() + parser.add_argument("--config_file_path", type=str, default="./config/NSFNet.yaml") + # args = parser.parse_args() + args = parser.parse_known_args()[0] + + config = load_yaml_config(args.config_file_path) + params = config["params"] + + model_name = params['model_name'] + case = params['case'] + device = params['device'] + device_id = params['device_id'] + network_size = params['network_size'] + learning_rate = params['learning_rate'] + epochs = params['epochs'] + batch_size = params['batch_size'] + load_params = params['load_params_test'] + second_path = params['second_path1'] + xmin = params['xmin'] + xmax = params['xmax'] + ymin = params['ymin'] + ymax = params['ymax'] + zmin = params['zmin'] + zmax = params['zmax'] + tmin = params['tmin'] + tmax = params['tmax'] + n_x = params['n_x'] + n_y = params['n_y'] + n_z = params['n_z'] + n_t = params['n_t'] + + os.environ['CUDA_VISIBLE_DEVICES'] = device_id + context.set_context(mode=context.GRAPH_MODE, save_graphs=False, device_target=device) + + use_ascend = context.get_context(attr_key='device_target') == "Ascend" + + if use_ascend: + msfloat_type = ms.float16 + npfloat_type = np.float16 + else: + msfloat_type = ms.float32 + npfloat_type = np.float32 + + x_b, y_b, z_b, t_b, x_i, y_i, z_i, t_i, u_b, v_b, w_b, \ + u_i, v_i, w_i, x_f, y_f, z_f, t_f, Re, X_min, X_max = read_training_data() + + model = VPNSFNets(x_b, y_b, z_b, t_b, x_i, y_i, z_i, t_i, u_b, v_b, w_b, \ + u_i, v_i, w_i, x_f, y_f, z_f, t_f, network_size, \ + Re, X_min, X_max, use_ascend, msfloat_type, npfloat_type, load_params, second_path) + + x_ts_sum = np.arange(xmin, xmax+1e-5, (xmax-xmin)/(n_x-1.)) + y_ts_sum = np.arange(ymin, ymax+1e-5, (ymax-ymin)/(n_y-1.)) + z_ts_sum = np.arange(zmin, zmax+1e-5, (zmax-zmin)/(n_z-1.)) + + x_ts, y_ts, z_ts = np.meshgrid(x_ts_sum, y_ts_sum, z_ts_sum) + x_ts = x_ts.reshape([-1, 1]) + y_ts = y_ts.reshape([-1, 1]) + z_ts = z_ts.reshape([-1, 1]) + x_ts = ms.Tensor(np.array(x_ts, npfloat_type)) + y_ts = ms.Tensor(np.array(y_ts, npfloat_type)) + z_ts = ms.Tensor(np.array(z_ts, npfloat_type)) + + # t1 + t1 = tmin*np.ones([x_ts.shape[0], 1]) + t1 = ms.Tensor(np.array(t1, npfloat_type)) + U_ts = model.net_u(x_ts, y_ts, z_ts, t1) + u_ts, v_ts, w_ts, p_ts = U_ts[:, 0:1], U_ts[:, 1:2], U_ts[:, 2:3], U_ts[:, 3:4] + u_ts_real, v_ts_real, w_ts_real, p_ts_real = \ + ThreeBeltramiflow(x_ts.numpy(), y_ts.numpy(), z_ts.numpy(), t1.numpy(), Re) + scipy.io.savemat('test_result1.mat', {'xts': x_ts.numpy(), 'yts': y_ts.numpy(), 'zts': z_ts.numpy(), + 'uts': u_ts.numpy(), 'vts': v_ts.numpy(), + 'wts': w_ts.numpy(), 'pts': p_ts.numpy(), + 'ureal': u_ts_real, 'vreal': v_ts_real, + 'wreal': w_ts_real, 'preal': p_ts_real}) + # t2 + t2 = (tmin + 0.25*(tmax-tmin))*np.ones([x_ts.shape[0], 1]) + t2 = ms.Tensor(np.array(t2, npfloat_type)) + U_ts = model.net_u(x_ts, y_ts, z_ts, t2) + u_ts, v_ts, w_ts, p_ts = U_ts[:, 0:1], U_ts[:, 1:2], U_ts[:, 2:3], U_ts[:, 3:4] + u_ts_real, v_ts_real, w_ts_real, p_ts_real = ThreeBeltramiflow(x_ts.numpy(), y_ts.numpy(), + z_ts.numpy(), t2.numpy(), Re) + scipy.io.savemat('test_result2.mat', {'xts': x_ts.numpy(), 'yts': y_ts.numpy(), + 'zts': z_ts.numpy(), 'uts': u_ts.numpy(), + 'vts': v_ts.numpy(), 'wts': w_ts.numpy(), + 'pts': p_ts.numpy(), 'ureal': u_ts_real, + 'vreal': v_ts_real, 'wreal': w_ts_real, + 'preal': p_ts_real}) + + # t3 + t3 = (tmin + 0.5*(tmax-tmin))*np.ones([x_ts.shape[0], 1]) + t3 = ms.Tensor(np.array(t3, npfloat_type)) + U_ts = model.net_u(x_ts, y_ts, z_ts, t3) + u_ts, v_ts, w_ts, p_ts = U_ts[:, 0:1], U_ts[:, 1:2], U_ts[:, 2:3], U_ts[:, 3:4] + u_ts_real, v_ts_real, w_ts_real, p_ts_real = ThreeBeltramiflow(x_ts.numpy(), y_ts.numpy(), + z_ts.numpy(), t3.numpy(), Re) + scipy.io.savemat('test_result3.mat', {'xts': x_ts.numpy(), 'yts': y_ts.numpy(), + 'zts': z_ts.numpy(), 'uts': u_ts.numpy(), + 'vts': v_ts.numpy(), 'wts': w_ts.numpy(), + 'pts': p_ts.numpy(), 'ureal': u_ts_real, + 'vreal': v_ts_real, 'wreal': w_ts_real, + 'preal': p_ts_real}) + + # t4 + t4 = (tmin + 0.75*(tmax-tmin))*np.ones([x_ts.shape[0], 1]) + t4 = ms.Tensor(np.array(t4, npfloat_type)) + U_ts = model.net_u(x_ts, y_ts, z_ts, t4) + u_ts, v_ts, w_ts, p_ts = U_ts[:, 0:1], U_ts[:, 1:2], U_ts[:, 2:3], U_ts[:, 3:4] + u_ts_real, v_ts_real, w_ts_real, p_ts_real = ThreeBeltramiflow(x_ts.numpy(), y_ts.numpy(), + z_ts.numpy(), t4.numpy(), Re) + scipy.io.savemat('test_result3.mat', {'xts': x_ts.numpy(), 'yts': y_ts.numpy(), + 'zts': z_ts.numpy(), 'uts': u_ts.numpy(), + 'vts': v_ts.numpy(), 'wts': w_ts.numpy(), + 'pts': p_ts.numpy(), 'ureal': u_ts_real, + 'vreal': v_ts_real, 'wreal': w_ts_real, + 'preal': p_ts_real}) + # t5 + t5 = tmax*np.ones([x_ts.shape[0], 1]) + t5 = ms.Tensor(np.array(t5, npfloat_type)) + U_ts = model.net_u(x_ts, y_ts, z_ts, t5) + u_ts, v_ts, w_ts, p_ts = U_ts[:, 0:1], U_ts[:, 1:2], U_ts[:, 2:3], U_ts[:, 3:4] + u_ts_real, v_ts_real, w_ts_real, p_ts_real = ThreeBeltramiflow(x_ts.numpy(), y_ts.numpy(), + z_ts.numpy(), t5.numpy(), Re) + scipy.io.savemat('test_result3.mat', {'xts': x_ts.numpy(), 'yts': y_ts.numpy(), + 'zts': z_ts.numpy(), 'uts': u_ts.numpy(), + 'vts': v_ts.numpy(), 'wts': w_ts.numpy(), + 'pts': p_ts.numpy(), 'ureal': u_ts_real, + 'vreal': v_ts_real, 'wreal': w_ts_real, + 'preal': p_ts_real}) diff --git a/MindFlow/applications/research/NSFNets/test_KF.py b/MindFlow/applications/research/NSFNets/test_KF.py new file mode 100644 index 0000000000000000000000000000000000000000..f08057f9f41124f055c96673949a7c6a29f7f5cf --- /dev/null +++ b/MindFlow/applications/research/NSFNets/test_KF.py @@ -0,0 +1,87 @@ +# Copyright 2023 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================ +"""NSFNet train""" +# pylint: disable=C0103 +import os +import argparse + +import numpy as np +import scipy.io +import mindspore as ms +from mindspore import set_seed, context +from mindflow.utils import load_yaml_config +from src.network_kf import VPNSFNets +from src.datasets_kf import read_training_data + +# Adam loss history +loss_history_adam_pretrain = np.empty([0]) +loss_b_history_adam_pretrain = np.empty([0]) +loss_f_history_adam_pretrain = np.empty([0]) +np.random.seed(123456) +set_seed(123456) + + +if __name__ == "__main__": + parser = argparse.ArgumentParser() + parser.add_argument("--config_file_path", type=str, default="./config/NSFNet_KF.yaml") + # args = parser.parse_args() + args = parser.parse_known_args()[0] + + config = load_yaml_config(args.config_file_path) + params = config["params"] + + model_name = params['model_name'] + case = params['case'] + device = params['device'] + device_id = params['device_id'] + network_size = params['network_size'] + learning_rate = params['learning_rate'] + epochs = params['epochs'] + load_params = params['load_params_test'] + second_path = params['second_path1'] + re = params['re'] + + os.environ['CUDA_VISIBLE_DEVICES'] = device_id + context.set_context(mode=context.GRAPH_MODE, save_graphs=False, device_target=device) + + use_ascend = context.get_context(attr_key='device_target') == "Ascend" + + if use_ascend: + msfloat_type = ms.float16 + npfloat_type = np.float16 + else: + msfloat_type = ms.float32 + npfloat_type = np.float32 + + xb_train, yb_train, ub_train, vb_train, x_train, y_train = read_training_data() + + model = VPNSFNets(xb_train, yb_train, ub_train, vb_train, x_train, y_train, \ + network_size, use_ascend, msfloat_type, npfloat_type, load_params, second_path) + + x_test = (np.random.rand(1000, 1) - 1 / 3) * 3 / 2 + y_test = (np.random.rand(1000, 1) - 1 / 4) * 2 + lam = 0.5 * re - np.sqrt(0.25 * (re ** 2) + 4 * (np.pi ** 2)) + u_test = 1 - np.exp(lam * x_test) * np.cos(2 * np.pi * y_test) + v_test = (lam / (2 * np.pi)) * np.exp(lam * x_test) * np.sin(2 * np.pi * y_test) + p_test = 0.5 * (1 - np.exp(2 * lam * x_test)) + # to tensor + x_test = ms.Tensor(np.array(x_test, npfloat_type)) + y_test = ms.Tensor(np.array(y_test, npfloat_type)) + + U_pred = model.net_u(x_test, y_test) + u_pred, v_pred, p_pred = U_pred[:, 0:1], U_pred[:, 1:2], U_pred[:, 2:3] + scipy.io.savemat('test_result.mat', {'xts': x_test.numpy(), 'yts': y_test.numpy(), \ + 'uts': u_pred.numpy(), 'vts': v_pred.numpy(), 'pts': p_pred.numpy(), \ + 'ureal': u_test, 'vreal': v_test, 'preal': p_test}) diff --git a/MindFlow/applications/research/NSFNets/train.py b/MindFlow/applications/research/NSFNets/train.py new file mode 100644 index 0000000000000000000000000000000000000000..d22e86febb8ec03378bfc041fc20118003c63979 --- /dev/null +++ b/MindFlow/applications/research/NSFNets/train.py @@ -0,0 +1,130 @@ +# Copyright 2023 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================ +"""ICNet train""" +# pylint: disable=C0103 +import os +import time +import argparse + +import numpy as np +import mindspore as ms +from mindspore import set_seed, context, nn, Tensor +from mindflow.utils import load_yaml_config +from src.network import VPNSFNets +from src.datasets import read_training_data + +# Adam loss history +loss_history_adam_pretrain = np.empty([0]) +loss_b_history_adam_pretrain = np.empty([0]) +loss_i_history_adam_pretrain = np.empty([0]) +loss_f_history_adam_pretrain = np.empty([0]) +np.random.seed(123456) +set_seed(123456) + +def train(model, niter, lr, batch_size): + """ + Training + """ + # Get the gradients function + params_train = model.dnn.trainable_params() + optimizer_adam = nn.Adam(params_train, learning_rate=lr) + grad_fn = ms.value_and_grad(model.loss_fn, None, optimizer_adam.parameters, has_aux=True) + model.dnn.set_train() + n_data = model.t_f.shape[0] + + start_time = time.time() + + for epoch in range(1, 1+niter): + + idx_data_0 = np.random.choice(n_data, batch_size) + idx_data = Tensor(idx_data_0) + x_batch = model.x_f[idx_data, :] + y_batch = model.y_f[idx_data, :] + z_batch = model.z_f[idx_data, :] + t_batch = model.t_f[idx_data, :] + + (loss, loss_b, loss_i, loss_f), grads = grad_fn(model.xb, model.yb, model.zb, model.tb, + model.xi, model.yi, model.zi, model.ti, + x_batch, y_batch, z_batch, t_batch, + model.ub, model.vb, model.wb, + model.ui, model.vi, model.wi) + optimizer_adam(grads) + + if epoch % 10 == 0: + elapsed = time.time() - start_time + print('It: %d, Total_loss: %.3e, Loss_b: %.3e, Loss_i: %.3e, Loss_f: %.3e, Time: %.2f' %\ + (epoch, loss.item(), loss_b.item(), loss_i.item(), loss_f.item(), elapsed)) + + global loss_history_adam_pretrain + global loss_b_history_adam_pretrain + global loss_i_history_adam_pretrain + global loss_f_history_adam_pretrain + + loss_history_adam_pretrain = np.append(loss_history_adam_pretrain, loss.numpy()) + loss_b_history_adam_pretrain = np.append(loss_b_history_adam_pretrain, loss_b.numpy()) + loss_i_history_adam_pretrain = np.append(loss_i_history_adam_pretrain, loss_i.numpy()) + loss_f_history_adam_pretrain = np.append(loss_f_history_adam_pretrain, loss_f.numpy()) + + start_time = time.time() + + +if __name__ == "__main__": + parser = argparse.ArgumentParser() + parser.add_argument("--config_file_path", type=str, default="./config/NSFNet.yaml") + # args = parser.parse_args() + args = parser.parse_known_args()[0] + + config = load_yaml_config(args.config_file_path) + params = config["params"] + + model_name = params['model_name'] + case = params['case'] + device = params['device'] + device_id = params['device_id'] + network_size = params['network_size'] + learning_rate = params['learning_rate'] + epochs = params['epochs'] + batch_size_train = params['batch_size'] + load_params = params['load_params'] + second_path = params['second_path1'] + re = params['re'] + + os.environ['CUDA_VISIBLE_DEVICES'] = device_id + context.set_context(mode=context.GRAPH_MODE, save_graphs=False, device_target=device) + + use_ascend = context.get_context(attr_key='device_target') == "Ascend" + + if use_ascend: + msfloat_type = ms.float16 + npfloat_type = np.float16 + else: + msfloat_type = ms.float32 + npfloat_type = np.float32 + + x_b, y_b, z_b, t_b, x_i, y_i, z_i, t_i, u_b, v_b, w_b, \ + u_i, v_i, w_i, x_f, y_f, z_f, t_f, X_min, X_max = read_training_data() + + model_train = VPNSFNets(x_b, y_b, z_b, t_b, x_i, y_i, z_i, t_i, u_b, v_b, w_b, + u_i, v_i, w_i, x_f, y_f, z_f, t_f, network_size, + re, X_min, X_max, use_ascend, msfloat_type, npfloat_type, load_params, second_path) + + for epoch_train, lr_train in zip(epochs, learning_rate): + train(model_train, int(np.float(epoch_train)), lr_train, batch_size=batch_size_train) + ms.save_checkpoint(model_train.dnn, f'model/{second_path}/model.ckpt') + + np.save(f'Loss-Coe/{second_path}/loss_history_Adam_pretrain', loss_history_adam_pretrain) + np.save(f'Loss-Coe/{second_path}/loss_b_history_Adam_pretrain', loss_b_history_adam_pretrain) + np.save(f'Loss-Coe/{second_path}/loss_i_history_Adam_pretrain', loss_i_history_adam_pretrain) + np.save(f'Loss-Coe/{second_path}/loss_f_history_Adam_pretrain', loss_f_history_adam_pretrain) diff --git a/MindFlow/applications/research/NSFNets/train_KF.py b/MindFlow/applications/research/NSFNets/train_KF.py new file mode 100644 index 0000000000000000000000000000000000000000..d9eb05cd11550a495ba0b2e9bd08a2d0c0f6bd42 --- /dev/null +++ b/MindFlow/applications/research/NSFNets/train_KF.py @@ -0,0 +1,110 @@ +# Copyright 2023 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================ +"""NSFNet train""" +# pylint: disable=C0103 +import os +import time +import argparse + +import numpy as np +import mindspore as ms +from mindspore import set_seed, context, nn +from mindflow.utils import load_yaml_config +from src.network_kf import VPNSFNets +from src.datasets_kf import read_training_data + +# Adam loss history +loss_history_adam_pretrain = np.empty([0]) +loss_b_history_adam_pretrain = np.empty([0]) +loss_f_history_adam_pretrain = np.empty([0]) +np.random.seed(123456) +set_seed(123456) + +def train(model, Niter, lr): + """ + Training + """ + # Get the gradients function + params_train = model.dnn.trainable_params() + optimizer_Adam = nn.Adam(params_train, learning_rate=lr) + grad_fn = ms.value_and_grad(model.loss_fn, None, optimizer_Adam.parameters, has_aux=True) + model.dnn.set_train() + + start_time = time.time() + + for epoch in range(1, 1+Niter): + + (loss, loss_b, loss_f), grads = grad_fn(model.xb, model.yb, model.x_f, model.y_f, model.ub, model.vb) + optimizer_Adam(grads) + + if epoch % 10 == 0: + elapsed = time.time() - start_time + print('It: %d, Total_loss: %.3e, Loss_b: %.3e, Loss_f: %.3e, Time: %.2f' %\ + (epoch, loss.item(), loss_b.item(), loss_f.item(), elapsed)) + + global loss_history_adam_pretrain + global loss_b_history_adam_pretrain + global loss_f_history_adam_pretrain + + loss_history_adam_pretrain = np.append(loss_history_adam_pretrain, loss.numpy()) + loss_b_history_adam_pretrain = np.append(loss_b_history_adam_pretrain, loss_b.numpy()) + loss_f_history_adam_pretrain = np.append(loss_f_history_adam_pretrain, loss_f.numpy()) + + start_time = time.time() + + +if __name__ == "__main__": + parser = argparse.ArgumentParser() + parser.add_argument("--config_file_path", type=str, default="./config/NSFNet_KF.yaml") + # args = parser.parse_args() + args = parser.parse_known_args()[0] + + config = load_yaml_config(args.config_file_path) + params = config["params"] + + model_name = params['model_name'] + case = params['case'] + device = params['device'] + device_id = params['device_id'] + network_size = params['network_size'] + learning_rate = params['learning_rate'] + epochs = params['epochs'] + load_params = params['load_params'] + second_path = params['second_path1'] + + os.environ['CUDA_VISIBLE_DEVICES'] = device_id + context.set_context(mode=context.GRAPH_MODE, save_graphs=False, device_target=device) + + use_ascend = context.get_context(attr_key='device_target') == "Ascend" + + if use_ascend: + msfloat_type = ms.float16 + npfloat_type = np.float16 + else: + msfloat_type = ms.float32 + npfloat_type = np.float32 + + xb_train, yb_train, ub_train, vb_train, x_train, y_train = read_training_data() + + model_train = VPNSFNets(xb_train, yb_train, ub_train, vb_train, x_train, + y_train, network_size, use_ascend, msfloat_type, + npfloat_type, load_params, second_path) + for epoch_train, lr_train in zip(epochs, learning_rate): + train(model_train, int(np.float(epoch_train)), lr_train) + ms.save_checkpoint(model_train.dnn, f'model/{second_path}/model.ckpt') + + np.save(f'Loss-Coe/{second_path}/loss_history_adam_pretrain', loss_history_adam_pretrain) + np.save(f'Loss-Coe/{second_path}/loss_b_history_adam_pretrain', loss_b_history_adam_pretrain) + np.save(f'Loss-Coe/{second_path}/loss_f_history_adam_pretrain', loss_f_history_adam_pretrain)