diff --git a/MindFlow/applications/research/ICNet/ICNet.ipynb b/MindFlow/applications/research/ICNet/ICNet.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..5b0901f027ed73f9ce88e0f601a6ec9e6903d69b --- /dev/null +++ b/MindFlow/applications/research/ICNet/ICNet.ipynb @@ -0,0 +1,434 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "de7a09e7-32fc-4059-9478-6c6fc6667f57", + "metadata": {}, + "source": [ + "# ICNet: Invariance Constrained Discovery for Partial Differential Equations\n", + "\n", + "## Environment Setup\n", + "\n", + "This notebook requires **MindSpore version >= 2.0.0** to support new APIs including: *mindspore.jit, mindspore.jit_class, mindspore.data_sink*. Please check [MindSpore Installation](https://www.mindspore.cn/install/en) for details.\n", + "\n", + "In addition, **MindFlow version >=0.1.0** is also required. If it has not been installed in your environment, please select the right version and hardware, then install it as follows." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "5a449343-731b-46bb-b8dd-f7a8c1ba8071", + "metadata": {}, + "outputs": [], + "source": [ + "mindflow_version = \"0.1.0\" # update if needed\n", + "# GPU Comment out the following code if you are using NPU.\n", + "!pip uninstall -y mindflow-gpu\n", + "!pip install mindflow-gpu==$mindflow_version\n", + "\n", + "# NPU Uncomment if needed.\n", + "# !pip uninstall -y mindflow-ascend\n", + "# !pip install mindflow-ascend==$mindflow_version" + ] + }, + { + "cell_type": "markdown", + "id": "25fe4081-9d91-44c1-b7de-b50a7645e090", + "metadata": {}, + "source": [ + "## Background\n", + "\n", + "The physical laws described by partial differential equations are widely present in the natural environment. The calculation and simulation of physical systems rely on accurate basic equations and models. The traditional method of deriving control equations is mainly based on first principles, such as the Navier-Stokes equations based on momentum conservation. The difficulty of traditional methods lies in the fact that models and equations of complex dynamics are often difficult to derive, such as multiphase flow, neuroscience, and biological science. In the era of big data, mining control equations from data through artificial intelligence methods has become a new research idea. The existing data-driven method of discovering equations still has certain limitations. At present, there is a lack of guiding principles when constructing candidates for a complete library, and it is impossible to ensure that the discovered equations meet basic physical requirements. At the same time, when dealing with complex multidimensional systems, the candidate library is too large, and it is difficult to discover simple and accurate equations. Considering that basic physical requirements (invariance, conservation, etc.) are the cornerstones of many physical problems, it is necessary to study how to impose physical constraints in discovering equations." + ] + }, + { + "cell_type": "markdown", + "id": "50540e43-3c8e-45f0-8e2b-78831ec89684", + "metadata": {}, + "source": [ + "## Model framework\n", + "\n", + "The model framework is as shown in the following figure:\n", + "\n", + "![ICNet](images/ICNet.png)\n", + "\n", + "In the figure:\n", + "A. Schematic diagram of the derivation process of embedding invariance constraints into the framework of partial differential equation discovery;\n", + "B. The neural network module of partial differential equation discovery with invariance constraints uses neural network automatic differentiation to obtain the partial derivatives required to construct the invariance candidate function. The loss function includes data loss, invariance loss, and regularization loss for enhancing sparsity.\n", + "\n", + "## Preparation\n", + "\n", + "Before practice, ensure that MindSpore of suitable version has been correctly installed. If not, you can run the following command:\n", + "\n", + "* [MindSpore installation page](https://www.mindspore.cn/install) Install MindSpore.\n", + "\n", + "## Datasets Preparation\n", + "\n", + "Dataset download link: [ICNet/dataset](https://download-mindspore.osinfra.cn/mindscience/mindflow/dataset/applications/research/ICNet/)。Save the dataset under path `./dataset`." + ] + }, + { + "cell_type": "markdown", + "id": "23990483-d613-4c8b-aa7e-0a8b16a89b13", + "metadata": {}, + "source": [ + "## Model Training\n", + "\n", + "Import code packs." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e3d17b8a-bde1-4ccd-9345-ccfaa2cd1bfe", + "metadata": {}, + "outputs": [], + "source": [ + "import argparse\n", + "import time\n", + "import numpy as np\n", + "\n", + "import mindspore as ms\n", + "from mindspore import set_seed, context, nn\n", + "from src.network import InvarianceConstrainedNN, InvarianceConstrainedNN_STRdige\n", + "from src.datasets import read_training_data, print_pde" + ] + }, + { + "cell_type": "markdown", + "id": "0b8e1d64-580b-4ae9-a7f6-164b0ee0adf3", + "metadata": {}, + "source": [ + "Setting of model-related parameters and definition of training model" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8a84c0ff-d4cf-41dd-8afc-db8f234d2108", + "metadata": {}, + "outputs": [], + "source": [ + "parser = argparse.ArgumentParser()\n", + "parser.add_argument('--model_name', type=str, default='ICNet')\n", + "parser.add_argument('--case', type=str, default='Kuramoto-Sivashinsky equation')\n", + "parser.add_argument('--device', type=str, default='GPU') #default='GPU' or 'Ascend'\n", + "parser.add_argument('--device_id', type=str, default=3)\n", + "parser.add_argument('--init_steps', type=str, default=0)\n", + "parser.add_argument('--stop_steps', type=str, default=150)\n", + "parser.add_argument('--time_steps', type=str, default=50)\n", + "parser.add_argument('--load_params', type=str, default='True')\n", + "parser.add_argument('--second_path', type=str, default='pretrain')\n", + "parser.add_argument('--data_name', type=str, default='KS.mat')\n", + "parser.add_argument('--description_ks', type=str, default=['uu_x', '1', 'u_x', 'u_xx', 'u_xxx', 'u_xxxx'])\n", + "parser.add_argument('--network_size', type=int, default=[2] + 8*[40] + [1])\n", + "parser.add_argument('--learning_rate', type=int, default=[0.001, 0.0005, 1.0e-04, 1.0e-05])\n", + "parser.add_argument('--epochs', type=int, default=[30e4, 30e4, 1e4, 1e4])\n", + "parser.add_argument('--BatchNo', type=int, default=1)\n", + "parser.add_argument('--lam', type=float, default=1e-5)\n", + "parser.add_argument('--d_tol', type=float, default=1.0)\n", + "args = parser.parse_known_args()[0]\n", + "\n", + "model_name = args.model_name\n", + "case = args.case\n", + "device = args.device\n", + "device_id = args.device_id\n", + "network_size = args.network_size\n", + "learning_rate = args.learning_rate\n", + "epochs = args.epochs\n", + "BatchNo = args.BatchNo\n", + "load_params = args.load_params\n", + "second_path = args.second_path\n", + "description_ks = args.description_ks\n", + "lam = args.lam\n", + "d_tol = args.d_tol\n", + "\n", + "use_ascend = context.get_context(attr_key='device_target') == \"Ascend\"\n", + "\n", + "if use_ascend:\n", + " msfloat_type = ms.float16\n", + "else:\n", + " msfloat_type = ms.float32\n", + "\n", + "context.set_context(mode=context.GRAPH_MODE, save_graphs=False, device_target=device, device_id=device_id)\n", + "\n", + "X_u_train, u_train, X_f_train = read_training_data(args)\n", + "\n", + "model_pretrain = InvarianceConstrainedNN(X_u_train, u_train, X_f_train, network_size, BatchNo, use_ascend, msfloat_type)" + ] + }, + { + "cell_type": "markdown", + "id": "387cb210-078e-41e1-b960-e432af9c8d5f", + "metadata": {}, + "source": [ + "Set the seed." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "482e625a-262a-4f06-a4a7-834766ed6ab4", + "metadata": {}, + "outputs": [], + "source": [ + "np.random.seed(123456)\n", + "set_seed(123456)" + ] + }, + { + "cell_type": "markdown", + "id": "30f8c191-ea21-4055-9d32-dfae30c96a33", + "metadata": {}, + "source": [ + "Code training and output results." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cff1733f-6e6a-47ff-928f-be3a348e98da", + "metadata": {}, + "outputs": [], + "source": [ + "def train(model, niter, lr):\n", + " # Get the gradients function\n", + " params = model.dnn.trainable_params()\n", + " params.append(model.lambda_u)\n", + " params.append(model.lambda_uux)\n", + "\n", + " optimizer_Adam = nn.Adam(params, learning_rate=lr)\n", + "\n", + " grad_fn = ms.value_and_grad(model.loss_fn, None, optimizer_Adam.parameters, has_aux=True)\n", + "\n", + " model.dnn.set_train()\n", + "\n", + " start_time = time.time()\n", + "\n", + " for epoch in range(1, 1+niter):\n", + " (loss, loss_u, loss_f_u, loss_lambda_u), grads = grad_fn(model.x, model.t, model.x_f, model.t_f, model.u)\n", + "\n", + " optimizer_Adam(grads)\n", + "\n", + " if epoch % 10 == 0:\n", + " elapsed = time.time() - start_time\n", + " print('It: %d, Loss: %.3e, loss_u: %.3e, loss_f: %.3e, loss_lambda: %.3e, Lambda_uux: %.3f, Lambda_uxx: %.3f, Lambda_uxxxx: %.3f, Time: %.2f' %\\\n", + " (epoch, loss.item(), loss_u.item(), loss_f_u.item(), loss_lambda_u.item(),\n", + " model.lambda_uux.item(), model.lambda_u[2].item(), model.lambda_u[4].item(), elapsed))\n", + "\n", + " initial_size = 5\n", + "\n", + " loss_history_Adam_Pretrain = np.empty([0])\n", + " loss_u_history_Adam_Pretrain = np.empty([0])\n", + " loss_f_u_history_Adam_Pretrain = np.empty([0])\n", + " loss_lambda_u_history_Adam_Pretrain = np.empty([0])\n", + "\n", + " lambda_u_history_Adam_Pretrain = np.zeros((initial_size, 1))\n", + " lambda_uux_history_Adam_Pretrain = np.zeros((1, 1))\n", + "\n", + " loss_history_Adam_Pretrain = np.append(loss_history_Adam_Pretrain, loss.numpy())\n", + " lambda_u_history_Adam_Pretrain = np.append(lambda_u_history_Adam_Pretrain, model.lambda_u.numpy(), axis=1)\n", + " loss_u_history_Adam_Pretrain = np.append(loss_u_history_Adam_Pretrain, loss_u.numpy())\n", + " loss_f_u_history_Adam_Pretrain = np.append(loss_f_u_history_Adam_Pretrain, loss_f_u.numpy())\n", + " loss_lambda_u_history_Adam_Pretrain = np.append(loss_lambda_u_history_Adam_Pretrain, loss_lambda_u.numpy())\n", + "\n", + " lambda_uux_new = np.array([model.lambda_uux.numpy()])\n", + " lambda_uux_history_Adam_Pretrain = np.append(lambda_uux_history_Adam_Pretrain, lambda_uux_new, axis=1)\n", + "\n", + " start_time = time.time()\n", + " np.save(f'Loss-Coe/{second_path}/loss_history_Adam_Pretrain', loss_history_Adam_Pretrain)\n", + " np.save(f'Loss-Coe/{second_path}/loss_u_history_Adam_Pretrain', loss_u_history_Adam_Pretrain)\n", + " np.save(f'Loss-Coe/{second_path}/loss_f_u_history_Adam_Pretrain', loss_f_u_history_Adam_Pretrain)\n", + " np.save(f'Loss-Coe/{second_path}/loss_lambda_u_history_Adam_Pretrain', loss_lambda_u_history_Adam_Pretrain)\n", + "\n", + " np.save(f'Loss-Coe/{second_path}/lambda_u_history_Adam_Pretrain', lambda_u_history_Adam_Pretrain)\n", + " np.save(f'Loss-Coe/{second_path}/lambda_uux_history_Adam_Pretrain', lambda_uux_history_Adam_Pretrain)" + ] + }, + { + "cell_type": "markdown", + "id": "cfedd05b-3d38-4156-b061-8278d232a7f9", + "metadata": {}, + "source": [ + "Run training and save the trained model." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c658395b-cade-408e-b999-585ba1ae73d5", + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "It: 0, Loss: 8.652e-01, loss_u: 8.652e-01, loss_f: 4.359e-08, loss_lambda: 0.000e+00, Lambda_uux: 0.000, Lambda_uxx: -0.000, Lambda_uxxxx: -0.000, Time: 9.59\n", + "It: 10, Loss: 8.434e-01, loss_u: 8.434e-01, loss_f: 7.940e-05, loss_lambda: 1.668e-09, Lambda_uux: 0.003, Lambda_uxx: 0.003, Lambda_uxxxx: -0.002, Time: 2.70\n", + "It: 20, Loss: 8.391e-01, loss_u: 8.390e-01, loss_f: 6.748e-05, loss_lambda: 4.801e-09, Lambda_uux: 0.012, Lambda_uxx: 0.011, Lambda_uxxxx: -0.006, Time: 2.71\n", + "It: 30, Loss: 8.277e-01, loss_u: 8.274e-01, loss_f: 3.814e-04, loss_lambda: 7.764e-09, Lambda_uux: 0.022, Lambda_uxx: 0.012, Lambda_uxxxx: -0.012, Time: 2.71\n", + "It: 40, Loss: 8.105e-01, loss_u: 8.096e-01, loss_f: 9.378e-04, loss_lambda: 1.053e-08, Lambda_uux: 0.034, Lambda_uxx: 0.001, Lambda_uxxxx: -0.025, Time: 2.70\n", + "It: 50, Loss: 7.876e-01, loss_u: 7.863e-01, loss_f: 1.300e-03, loss_lambda: 1.481e-08, Lambda_uux: 0.047, Lambda_uxx: -0.014, Lambda_uxxxx: -0.024, Time: 2.70\n", + "It: 60, Loss: 7.637e-01, loss_u: 7.625e-01, loss_f: 1.200e-03, loss_lambda: 1.795e-08, Lambda_uux: 0.050, Lambda_uxx: -0.031, Lambda_uxxxx: -0.010, Time: 2.71\n", + "It: 70, Loss: 7.529e-01, loss_u: 7.517e-01, loss_f: 1.191e-03, loss_lambda: 2.110e-08, Lambda_uux: 0.040, Lambda_uxx: -0.043, Lambda_uxxxx: -0.005, Time: 2.71\n", + "It: 80, Loss: 7.427e-01, loss_u: 7.414e-01, loss_f: 1.279e-03, loss_lambda: 2.508e-08, Lambda_uux: 0.025, Lambda_uxx: -0.048, Lambda_uxxxx: -0.013, Time: 2.72\n", + "It: 90, Loss: 7.363e-01, loss_u: 7.349e-01, loss_f: 1.400e-03, loss_lambda: 2.884e-08, Lambda_uux: 0.008, Lambda_uxx: -0.049, Lambda_uxxxx: -0.021, Time: 2.73\n", + "It: 100, Loss: 7.387e-01, loss_u: 7.375e-01, loss_f: 1.260e-03, loss_lambda: 3.227e-08, Lambda_uux: -0.011, Lambda_uxx: -0.052, Lambda_uxxxx: -0.020, Time: 2.72\n", + "It: 110, Loss: 6.951e-01, loss_u: 6.938e-01, loss_f: 1.347e-03, loss_lambda: 3.469e-08, Lambda_uux: -0.029, Lambda_uxx: -0.050, Lambda_uxxxx: -0.016, Time: 2.71\n", + "It: 120, Loss: 6.755e-01, loss_u: 6.739e-01, loss_f: 1.603e-03, loss_lambda: 3.688e-08, Lambda_uux: -0.048, Lambda_uxx: -0.041, Lambda_uxxxx: -0.010, Time: 2.73\n", + "It: 130, Loss: 6.711e-01, loss_u: 6.691e-01, loss_f: 2.080e-03, loss_lambda: 3.856e-08, Lambda_uux: -0.067, Lambda_uxx: -0.031, Lambda_uxxxx: -0.008, Time: 2.73\n", + "It: 140, Loss: 6.587e-01, loss_u: 6.566e-01, loss_f: 2.077e-03, loss_lambda: 4.122e-08, Lambda_uux: -0.085, Lambda_uxx: -0.030, Lambda_uxxxx: -0.010, Time: 2.73\n", + "It: 150, Loss: 6.607e-01, loss_u: 6.589e-01, loss_f: 1.852e-03, loss_lambda: 4.403e-08, Lambda_uux: -0.103, Lambda_uxx: -0.038, Lambda_uxxxx: -0.010, Time: 2.72\n", + "It: 160, Loss: 6.526e-01, loss_u: 6.509e-01, loss_f: 1.731e-03, loss_lambda: 4.603e-08, Lambda_uux: -0.119, Lambda_uxx: -0.041, Lambda_uxxxx: -0.011, Time: 2.74\n", + "It: 170, Loss: 6.486e-01, loss_u: 6.468e-01, loss_f: 1.820e-03, loss_lambda: 4.645e-08, Lambda_uux: -0.133, Lambda_uxx: -0.039, Lambda_uxxxx: -0.008, Time: 2.75\n", + "It: 180, Loss: 6.398e-01, loss_u: 6.379e-01, loss_f: 1.904e-03, loss_lambda: 4.652e-08, Lambda_uux: -0.142, Lambda_uxx: -0.037, Lambda_uxxxx: -0.008, Time: 2.75\n", + "It: 190, Loss: 6.331e-01, loss_u: 6.311e-01, loss_f: 1.962e-03, loss_lambda: 4.687e-08, Lambda_uux: -0.147, Lambda_uxx: -0.041, Lambda_uxxxx: -0.008, Time: 2.75\n", + "It: 200, Loss: 6.274e-01, loss_u: 6.253e-01, loss_f: 2.085e-03, loss_lambda: 4.720e-08, Lambda_uux: -0.148, Lambda_uxx: -0.045, Lambda_uxxxx: -0.010, Time: 2.75\n", + "It: 210, Loss: 6.536e-01, loss_u: 6.519e-01, loss_f: 1.771e-03, loss_lambda: 4.710e-08, Lambda_uux: -0.148, Lambda_uxx: -0.046, Lambda_uxxxx: -0.011, Time: 2.77\n", + "It: 220, Loss: 6.334e-01, loss_u: 6.315e-01, loss_f: 1.926e-03, loss_lambda: 4.675e-08, Lambda_uux: -0.148, Lambda_uxx: -0.042, Lambda_uxxxx: -0.010, Time: 2.77\n", + "It: 230, Loss: 6.245e-01, loss_u: 6.225e-01, loss_f: 1.961e-03, loss_lambda: 4.706e-08, Lambda_uux: -0.151, Lambda_uxx: -0.042, Lambda_uxxxx: -0.009, Time: 2.77\n", + "It: 240, Loss: 6.263e-01, loss_u: 6.241e-01, loss_f: 2.160e-03, loss_lambda: 4.861e-08, Lambda_uux: -0.156, Lambda_uxx: -0.048, Lambda_uxxxx: -0.009, Time: 2.78\n", + "It: 250, Loss: 6.191e-01, loss_u: 6.170e-01, loss_f: 2.121e-03, loss_lambda: 4.961e-08, Lambda_uux: -0.160, Lambda_uxx: -0.047, Lambda_uxxxx: -0.013, Time: 2.79\n", + "It: 260, Loss: 6.183e-01, loss_u: 6.161e-01, loss_f: 2.112e-03, loss_lambda: 4.981e-08, Lambda_uux: -0.163, Lambda_uxx: -0.046, Lambda_uxxxx: -0.011, Time: 2.78\n", + "It: 270, Loss: 6.363e-01, loss_u: 6.348e-01, loss_f: 1.559e-03, loss_lambda: 4.977e-08, Lambda_uux: -0.165, Lambda_uxx: -0.046, Lambda_uxxxx: -0.009, Time: 2.79\n", + "It: 280, Loss: 6.233e-01, loss_u: 6.219e-01, loss_f: 1.398e-03, loss_lambda: 4.956e-08, Lambda_uux: -0.164, Lambda_uxx: -0.039, Lambda_uxxxx: -0.011, Time: 2.76\n", + "It: 290, Loss: 6.165e-01, loss_u: 6.149e-01, loss_f: 1.583e-03, loss_lambda: 4.968e-08, Lambda_uux: -0.164, Lambda_uxx: -0.041, Lambda_uxxxx: -0.009, Time: 2.78\n", + "It: 300, Loss: 6.152e-01, loss_u: 6.133e-01, loss_f: 1.894e-03, loss_lambda: 5.086e-08, Lambda_uux: -0.166, Lambda_uxx: -0.045, Lambda_uxxxx: -0.010, Time: 2.79\n", + "It: 310, Loss: 6.131e-01, loss_u: 6.111e-01, loss_f: 1.962e-03, loss_lambda: 5.149e-08, Lambda_uux: -0.168, Lambda_uxx: -0.044, Lambda_uxxxx: -0.012, Time: 2.76\n", + "It: 320, Loss: 6.118e-01, loss_u: 6.099e-01, loss_f: 1.951e-03, loss_lambda: 5.119e-08, Lambda_uux: -0.167, Lambda_uxx: -0.042, Lambda_uxxxx: -0.010, Time: 2.77\n", + "It: 330, Loss: 6.117e-01, loss_u: 6.098e-01, loss_f: 1.895e-03, loss_lambda: 5.083e-08, Lambda_uux: -0.164, Lambda_uxx: -0.040, Lambda_uxxxx: -0.009, Time: 2.83\n", + "It: 340, Loss: 6.189e-01, loss_u: 6.172e-01, loss_f: 1.712e-03, loss_lambda: 5.067e-08, Lambda_uux: -0.160, Lambda_uxx: -0.039, Lambda_uxxxx: -0.009, Time: 2.78\n", + "It: 350, Loss: 6.082e-01, loss_u: 6.065e-01, loss_f: 1.698e-03, loss_lambda: 5.041e-08, Lambda_uux: -0.155, Lambda_uxx: -0.037, Lambda_uxxxx: -0.008, Time: 2.77\n", + "It: 360, Loss: 6.081e-01, loss_u: 6.063e-01, loss_f: 1.809e-03, loss_lambda: 5.056e-08, Lambda_uux: -0.152, Lambda_uxx: -0.036, Lambda_uxxxx: -0.009, Time: 2.79\n", + "It: 370, Loss: 6.051e-01, loss_u: 6.032e-01, loss_f: 1.876e-03, loss_lambda: 5.059e-08, Lambda_uux: -0.148, Lambda_uxx: -0.036, Lambda_uxxxx: -0.008, Time: 2.79\n", + "It: 380, Loss: 6.055e-01, loss_u: 6.037e-01, loss_f: 1.803e-03, loss_lambda: 5.015e-08, Lambda_uux: -0.143, Lambda_uxx: -0.035, Lambda_uxxxx: -0.006, Time: 2.77\n", + "It: 390, Loss: 6.034e-01, loss_u: 6.016e-01, loss_f: 1.792e-03, loss_lambda: 4.962e-08, Lambda_uux: -0.138, Lambda_uxx: -0.034, Lambda_uxxxx: -0.006, Time: 2.77\n", + "It: 400, Loss: 6.059e-01, loss_u: 6.041e-01, loss_f: 1.862e-03, loss_lambda: 4.929e-08, Lambda_uux: -0.134, Lambda_uxx: -0.034, Lambda_uxxxx: -0.006, Time: 2.77\n", + "It: 410, Loss: 6.145e-01, loss_u: 6.125e-01, loss_f: 1.953e-03, loss_lambda: 4.881e-08, Lambda_uux: -0.130, Lambda_uxx: -0.033, Lambda_uxxxx: -0.006, Time: 2.79\n", + "It: 420, Loss: 6.006e-01, loss_u: 5.988e-01, loss_f: 1.864e-03, loss_lambda: 4.812e-08, Lambda_uux: -0.126, Lambda_uxx: -0.032, Lambda_uxxxx: -0.006, Time: 2.78\n", + "It: 430, Loss: 6.028e-01, loss_u: 6.009e-01, loss_f: 1.912e-03, loss_lambda: 4.785e-08, Lambda_uux: -0.123, Lambda_uxx: -0.031, Lambda_uxxxx: -0.006, Time: 2.77\n", + "It: 440, Loss: 6.053e-01, loss_u: 6.034e-01, loss_f: 1.977e-03, loss_lambda: 4.752e-08, Lambda_uux: -0.120, Lambda_uxx: -0.031, Lambda_uxxxx: -0.005, Time: 2.78\n", + "It: 450, Loss: 6.020e-01, loss_u: 6.000e-01, loss_f: 2.001e-03, loss_lambda: 4.713e-08, Lambda_uux: -0.118, Lambda_uxx: -0.030, Lambda_uxxxx: -0.005, Time: 2.77\n", + "It: 460, Loss: 6.029e-01, loss_u: 6.010e-01, loss_f: 1.983e-03, loss_lambda: 4.692e-08, Lambda_uux: -0.114, Lambda_uxx: -0.029, Lambda_uxxxx: -0.004, Time: 2.76\n", + "It: 470, Loss: 6.003e-01, loss_u: 5.984e-01, loss_f: 1.935e-03, loss_lambda: 4.677e-08, Lambda_uux: -0.111, Lambda_uxx: -0.029, Lambda_uxxxx: -0.004, Time: 2.82\n", + "It: 480, Loss: 5.998e-01, loss_u: 5.978e-01, loss_f: 2.000e-03, loss_lambda: 4.657e-08, Lambda_uux: -0.108, Lambda_uxx: -0.028, Lambda_uxxxx: -0.003, Time: 2.85\n", + "It: 490, Loss: 5.983e-01, loss_u: 5.962e-01, loss_f: 2.034e-03, loss_lambda: 4.634e-08, Lambda_uux: -0.103, Lambda_uxx: -0.027, Lambda_uxxxx: -0.003, Time: 2.78\n", + "It: 500, Loss: 6.057e-01, loss_u: 6.037e-01, loss_f: 1.992e-03, loss_lambda: 4.605e-08, Lambda_uux: -0.097, Lambda_uxx: -0.026, Lambda_uxxxx: -0.003, Time: 2.77\n", + "It: 510, Loss: 5.983e-01, loss_u: 5.964e-01, loss_f: 1.930e-03, loss_lambda: 4.612e-08, Lambda_uux: -0.091, Lambda_uxx: -0.026, Lambda_uxxxx: -0.003, Time: 2.77\n", + "It: 520, Loss: 5.936e-01, loss_u: 5.915e-01, loss_f: 2.025e-03, loss_lambda: 4.610e-08, Lambda_uux: -0.087, Lambda_uxx: -0.026, Lambda_uxxxx: -0.003, Time: 2.76\n", + "It: 530, Loss: 5.977e-01, loss_u: 5.956e-01, loss_f: 2.026e-03, loss_lambda: 4.607e-08, Lambda_uux: -0.081, Lambda_uxx: -0.025, Lambda_uxxxx: -0.002, Time: 2.76\n", + "It: 540, Loss: 5.984e-01, loss_u: 5.964e-01, loss_f: 2.023e-03, loss_lambda: 4.599e-08, Lambda_uux: -0.074, Lambda_uxx: -0.026, Lambda_uxxxx: -0.002, Time: 2.78\n", + "It: 550, Loss: 5.955e-01, loss_u: 5.935e-01, loss_f: 1.976e-03, loss_lambda: 4.590e-08, Lambda_uux: -0.068, Lambda_uxx: -0.026, Lambda_uxxxx: -0.002, Time: 2.79\n", + "It: 560, Loss: 5.927e-01, loss_u: 5.907e-01, loss_f: 1.977e-03, loss_lambda: 4.610e-08, Lambda_uux: -0.063, Lambda_uxx: -0.026, Lambda_uxxxx: -0.002, Time: 2.78\n" + ] + } + ], + "source": [ + "for epoch, lr in zip(epochs, learning_rate):\n", + " train(model_pretrain, int(epoch), lr)\n", + "ms.save_checkpoint(model_pretrain.dnn, f'model/{second_path}/model.ckpt')" + ] + }, + { + "cell_type": "markdown", + "id": "60d9233f-3748-4481-9d91-0c6c4ff42d26", + "metadata": {}, + "source": [ + "Save the learnable parameters of the last training for equation discovery." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b2ba9195-6d20-46ef-8bb4-f77e00914aae", + "metadata": {}, + "outputs": [], + "source": [ + "lambda_uux_value = model_pretrain.lambda_uux.numpy()\n", + "lambda_u_value = model_pretrain.lambda_u.numpy()\n", + "np.save(f'Loss-Coe/{second_path}/lambda_uux_value', lambda_uux_value)\n", + "np.save(f'Loss-Coe/{second_path}/lambda_u_value', lambda_u_value)" + ] + }, + { + "cell_type": "markdown", + "id": "a5f276c8-9b5a-4f01-a76e-90e4c5addff9", + "metadata": {}, + "source": [ + "Directly performing equation discovery after training may exceed the memory of GPU or NPU, so it is necessary to determine whether to reload the model for equation discovery based on the GPU or NPU memory size." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1b672fa0-3903-4ac7-9395-211895cfaeba", + "metadata": {}, + "outputs": [], + "source": [ + "if load_params:\n", + " lambda_u_value = np.load(f'Loss-Coe/{second_path}/lambda_u_value.npy')\n", + " lambda_uux_value = np.load(f'Loss-Coe/{second_path}/lambda_uux_value.npy')\n", + " model_ICCO = InvarianceConstrainedNN_STRdige(X_u_train, u_train, X_f_train, network_size, BatchNo, lambda_u_value, lambda_uux_value, load_params, second_path, msfloat_type)\n", + "else:\n", + " model_ICCO = InvarianceConstrainedNN_STRdige(X_u_train, u_train, X_f_train, network_size, BatchNo, lambda_u_value, lambda_uux_value, load_params, second_path, msfloat_type)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1b8887fa-9550-4873-a5cf-edd2a49b76ff", + "metadata": { + "scrolled": true + }, + "outputs": [], + "source": [ + "lambda_u_STRidge = model_ICCO.call_trainstridge(lam, d_tol)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "fdb0acd9-8890-4bb9-bde1-77cf4e9b17f3", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "u_t = (0.973947)uu_x\n", + " (-0.967219)u_xx\n", + " + (-0.967183)u_xxxx\n", + " \n" + ] + } + ], + "source": [ + "# GPU results\n", + "print_pde(lambda_uux_value, lambda_u_STRidge, description_ks, ut='u_t')" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.19" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/MindFlow/applications/research/ICNet/ICNet_CN.ipynb b/MindFlow/applications/research/ICNet/ICNet_CN.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..fcf97b7927fbf8002807615eb44be26ffd52c4d4 --- /dev/null +++ b/MindFlow/applications/research/ICNet/ICNet_CN.ipynb @@ -0,0 +1,448 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "de7a09e7-32fc-4059-9478-6c6fc6667f57", + "metadata": {}, + "source": [ + "# ICNet 不变性约束发现偏微分方程\n", + "\n", + "## 环境安装\n", + "\n", + "本案例要求 MindSpore >= 2.0.0 版本以调用如下接口: mindspore.jit, mindspore.jit_class, mindspore.data_sink。具体请查看MindSpore安装。\n", + "\n", + "此外，你需要安装 MindFlow >=0.1.0 版本。如果当前环境还没有安装，请按照下列方式选择后端和版本进行安装。" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "5a449343-731b-46bb-b8dd-f7a8c1ba8071", + "metadata": {}, + "outputs": [], + "source": [ + "mindflow_version = \"0.1.0\" # update if needed\n", + "# GPU Comment out the following code if you are using NPU.\n", + "!pip uninstall -y mindflow-gpu\n", + "!pip install mindflow-gpu==$mindflow_version\n", + "\n", + "# NPU Uncomment if needed.\n", + "# !pip uninstall -y mindflow-ascend\n", + "# !pip install mindflow-ascend==$mindflow_version" + ] + }, + { + "cell_type": "markdown", + "id": "25fe4081-9d91-44c1-b7de-b50a7645e090", + "metadata": {}, + "source": [ + "## 背景介绍\n", + "\n", + "由偏微分方程描述的物理规律广泛存在于自然环境之中，物理系统的计算与模拟依赖于准确的基本方程和模型，传统方法推导控制方程主要基于第一性原理，例如Navier-Stokes方程基于动量守恒，传统方法难点在于复杂动力学的模型与方程常常难以推导，例如多相流、神经科学以及生物科学等，在大数据时代，通过人工智能的方法从数据中挖掘控制方程成为一种新的研究思路。已有的数据驱动发现方程的方法依然存在一定的局限性，目前构建过完备库的候选项时缺乏指导原则，无法保证发现的方程满足基本的物理要求，同时在处理复杂多维系统时候选库过大，而难以发现出简约准确的方程。考虑到基本的物理要求（不变性，守恒性等）是很多物理问题出发的基石，因此有必要研究如何在发现方程中施加物理约束。" + ] + }, + { + "cell_type": "markdown", + "id": "50540e43-3c8e-45f0-8e2b-78831ec89684", + "metadata": {}, + "source": [ + "## 模型框架\n", + "\n", + "模型框架图如下所示:\n", + "\n", + "![ICNet](images/ICNet.png)\n", + "\n", + "图中：\n", + "A. 嵌入不变性约束至发现偏微分方程框架中的推导过程示意图；\n", + "B. 不变性约束发现偏微分方程的神经网络模块，利用神经网络自动微分求出构建不变性候选函数所需要的偏导数，损失函数包括数据损失Data loss，不变性损失Invariance loss以及增强稀疏性的正则化损失Regularization loss。\n", + "\n", + "## 准备环节\n", + "\n", + "实践前，确保已经正确安装合适版本的MindSpore。如果没有，可以通过：\n", + "\n", + "* [MindSpore安装页面](https://www.mindspore.cn/install) 安装MindSpore。\n", + "\n", + "## 数据集的准备\n", + "\n", + "数据集下载地址：[ICNet/dataset](https://download-mindspore.osinfra.cn/mindscience/mindflow/dataset/applications/research/ICNet/)。将数据集保存在`./dataset`路径下。" + ] + }, + { + "cell_type": "markdown", + "id": "23990483-d613-4c8b-aa7e-0a8b16a89b13", + "metadata": {}, + "source": [ + "## 模型训练\n", + "\n", + "引入代码包" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e3d17b8a-bde1-4ccd-9345-ccfaa2cd1bfe", + "metadata": {}, + "outputs": [], + "source": [ + "import argparse\n", + "import time\n", + "import numpy as np\n", + "\n", + "import mindspore as ms\n", + "from mindspore import set_seed, context, nn\n", + "from src.network import InvarianceConstrainedNN, InvarianceConstrainedNN_STRdige\n", + "from src.datasets import read_training_data, print_pde" + ] + }, + { + "cell_type": "markdown", + "id": "0b8e1d64-580b-4ae9-a7f6-164b0ee0adf3", + "metadata": {}, + "source": [ + "模型相关参数的设置以及训练模型的定义" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8a84c0ff-d4cf-41dd-8afc-db8f234d2108", + "metadata": {}, + "outputs": [], + "source": [ + "parser = argparse.ArgumentParser()\n", + "parser.add_argument('--model_name', type=str, default='ICNet')\n", + "parser.add_argument('--case', type=str, default='Kuramoto-Sivashinsky equation')\n", + "parser.add_argument('--device', type=str, default='GPU') #default='GPU' or 'Ascend'\n", + "parser.add_argument('--device_id', type=str, default=3)\n", + "parser.add_argument('--init_steps', type=str, default=0)\n", + "parser.add_argument('--stop_steps', type=str, default=150)\n", + "parser.add_argument('--time_steps', type=str, default=50)\n", + "parser.add_argument('--load_params', type=str, default='True')\n", + "parser.add_argument('--second_path', type=str, default='pretrain')\n", + "parser.add_argument('--data_name', type=str, default='KS.mat')\n", + "parser.add_argument('--description_ks', type=str, default=['uu_x', '1', 'u_x', 'u_xx', 'u_xxx', 'u_xxxx'])\n", + "parser.add_argument('--network_size', type=int, default=[2] + 8*[40] + [1])\n", + "parser.add_argument('--learning_rate', type=int, default=[0.001, 0.0005, 1.0e-04, 1.0e-05])\n", + "parser.add_argument('--epochs', type=int, default=[30e4, 30e4, 1e4, 1e4])\n", + "parser.add_argument('--BatchNo', type=int, default=1)\n", + "parser.add_argument('--lam', type=float, default=1e-5)\n", + "parser.add_argument('--d_tol', type=float, default=1.0)\n", + "args = parser.parse_known_args()[0]\n", + "\n", + "model_name = args.model_name\n", + "case = args.case\n", + "device = args.device\n", + "device_id = args.device_id\n", + "network_size = args.network_size\n", + "learning_rate = args.learning_rate\n", + "epochs = args.epochs\n", + "BatchNo = args.BatchNo\n", + "load_params = args.load_params\n", + "second_path = args.second_path\n", + "description_ks = args.description_ks\n", + "lam = args.lam\n", + "d_tol = args.d_tol\n", + "\n", + "use_ascend = context.get_context(attr_key='device_target') == \"Ascend\"\n", + "\n", + "if use_ascend:\n", + " msfloat_type = ms.float16\n", + "else:\n", + " msfloat_type = ms.float32\n", + "\n", + "context.set_context(mode=context.GRAPH_MODE, save_graphs=False, device_target=device, device_id=device_id)\n", + "\n", + "X_u_train, u_train, X_f_train = read_training_data(args)\n", + "\n", + "model_pretrain = InvarianceConstrainedNN(X_u_train, u_train, X_f_train, network_size, BatchNo, use_ascend, msfloat_type)" + ] + }, + { + "cell_type": "markdown", + "id": "387cb210-078e-41e1-b960-e432af9c8d5f", + "metadata": {}, + "source": [ + "设置种子" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "482e625a-262a-4f06-a4a7-834766ed6ab4", + "metadata": {}, + "outputs": [], + "source": [ + "np.random.seed(123456)\n", + "set_seed(123456)" + ] + }, + { + "cell_type": "markdown", + "id": "30f8c191-ea21-4055-9d32-dfae30c96a33", + "metadata": {}, + "source": [ + "代码训练与输出结果部分" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cff1733f-6e6a-47ff-928f-be3a348e98da", + "metadata": {}, + "outputs": [], + "source": [ + "def train(model, niter, lr):\n", + " # Get the gradients function\n", + "\n", + " params = model.dnn.trainable_params()\n", + " params.append(model.lambda_u)\n", + " params.append(model.lambda_uux)\n", + "\n", + " optimizer_Adam = nn.Adam(params, learning_rate=lr)\n", + "\n", + " grad_fn = ms.value_and_grad(model.loss_fn, None, optimizer_Adam.parameters, has_aux=True)\n", + "\n", + " model.dnn.set_train()\n", + "\n", + " start_time = time.time()\n", + "\n", + " for epoch in range(1, 1+niter):\n", + " (loss, loss_u, loss_f_u, loss_lambda_u), grads = grad_fn(model.x, model.t, model.x_f, model.t_f, model.u)\n", + "\n", + " optimizer_Adam(grads)\n", + "\n", + " if epoch % 10 == 0:\n", + " elapsed = time.time() - start_time\n", + " print('It: %d, Loss: %.3e, loss_u: %.3e, loss_f: %.3e, loss_lambda: %.3e, Lambda_uux: %.3f, Lambda_uxx: %.3f, Lambda_uxxxx: %.3f, Time: %.2f' %\\\n", + " (epoch, loss.item(), loss_u.item(), loss_f_u.item(), loss_lambda_u.item(),\n", + " model.lambda_uux.item(), model.lambda_u[2].item(), model.lambda_u[4].item(), elapsed))\n", + "\n", + " initial_size = 5\n", + "\n", + " loss_history_Adam_Pretrain = np.empty([0])\n", + " loss_u_history_Adam_Pretrain = np.empty([0])\n", + " loss_f_u_history_Adam_Pretrain = np.empty([0])\n", + " loss_lambda_u_history_Adam_Pretrain = np.empty([0])\n", + "\n", + " lambda_u_history_Adam_Pretrain = np.zeros((initial_size, 1))\n", + " lambda_uux_history_Adam_Pretrain = np.zeros((1, 1))\n", + "\n", + " loss_history_Adam_Pretrain = np.append(loss_history_Adam_Pretrain, loss.numpy())\n", + " lambda_u_history_Adam_Pretrain = np.append(lambda_u_history_Adam_Pretrain, model.lambda_u.numpy(), axis=1)\n", + " loss_u_history_Adam_Pretrain = np.append(loss_u_history_Adam_Pretrain, loss_u.numpy())\n", + " loss_f_u_history_Adam_Pretrain = np.append(loss_f_u_history_Adam_Pretrain, loss_f_u.numpy())\n", + " loss_lambda_u_history_Adam_Pretrain = np.append(loss_lambda_u_history_Adam_Pretrain, loss_lambda_u.numpy())\n", + "\n", + " lambda_uux_new = np.array([model.lambda_uux.numpy()])\n", + " lambda_uux_history_Adam_Pretrain = np.append(lambda_uux_history_Adam_Pretrain, lambda_uux_new, axis=1)\n", + "\n", + " start_time = time.time()\n", + " np.save(f'Loss-Coe/{second_path}/loss_history_Adam_Pretrain', loss_history_Adam_Pretrain)\n", + " np.save(f'Loss-Coe/{second_path}/loss_u_history_Adam_Pretrain', loss_u_history_Adam_Pretrain)\n", + " np.save(f'Loss-Coe/{second_path}/loss_f_u_history_Adam_Pretrain', loss_f_u_history_Adam_Pretrain)\n", + " np.save(f'Loss-Coe/{second_path}/loss_lambda_u_history_Adam_Pretrain', loss_lambda_u_history_Adam_Pretrain)\n", + "\n", + " np.save(f'Loss-Coe/{second_path}/lambda_u_history_Adam_Pretrain', lambda_u_history_Adam_Pretrain)\n", + " np.save(f'Loss-Coe/{second_path}/lambda_uux_history_Adam_Pretrain', lambda_uux_history_Adam_Pretrain)" + ] + }, + { + "cell_type": "markdown", + "id": "cfedd05b-3d38-4156-b061-8278d232a7f9", + "metadata": {}, + "source": [ + "运行训练与保存训练模型" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c658395b-cade-408e-b999-585ba1ae73d5", + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "It: 0, Loss: 1.059e+00, loss_u: 1.059e+00, loss_f: 2.203e-07, loss_lambda: 0.000e+00, Lambda_uux: -0.000, Lambda_uxx: -0.000, Lambda_uxxxx: -0.000, Time: 9.18\n", + "It: 10, Loss: 1.045e+00, loss_u: 1.045e+00, loss_f: 4.615e-05, loss_lambda: 1.824e-09, Lambda_uux: 0.004, Lambda_uxx: 0.003, Lambda_uxxxx: -0.003, Time: 2.72\n", + "It: 20, Loss: 1.040e+00, loss_u: 1.039e+00, loss_f: 1.004e-04, loss_lambda: 3.732e-09, Lambda_uux: 0.011, Lambda_uxx: 0.005, Lambda_uxxxx: 0.001, Time: 2.72\n", + "It: 30, Loss: 1.034e+00, loss_u: 1.034e+00, loss_f: 2.701e-04, loss_lambda: 6.927e-09, Lambda_uux: 0.004, Lambda_uxx: -0.005, Lambda_uxxxx: 0.009, Time: 2.70\n", + "It: 40, Loss: 1.029e+00, loss_u: 1.028e+00, loss_f: 6.826e-04, loss_lambda: 1.216e-08, Lambda_uux: -0.009, Lambda_uxx: -0.019, Lambda_uxxxx: 0.023, Time: 2.73\n", + "It: 50, Loss: 1.022e+00, loss_u: 1.021e+00, loss_f: 1.178e-03, loss_lambda: 1.785e-08, Lambda_uux: -0.020, Lambda_uxx: -0.033, Lambda_uxxxx: 0.035, Time: 2.70\n", + "It: 60, Loss: 1.017e+00, loss_u: 1.016e+00, loss_f: 1.510e-03, loss_lambda: 2.169e-08, Lambda_uux: -0.022, Lambda_uxx: -0.041, Lambda_uxxxx: 0.031, Time: 2.70\n", + "It: 70, Loss: 1.011e+00, loss_u: 1.010e+00, loss_f: 1.533e-03, loss_lambda: 2.363e-08, Lambda_uux: -0.020, Lambda_uxx: -0.039, Lambda_uxxxx: 0.019, Time: 2.71\n", + "It: 80, Loss: 1.005e+00, loss_u: 1.003e+00, loss_f: 1.808e-03, loss_lambda: 2.458e-08, Lambda_uux: -0.025, Lambda_uxx: -0.025, Lambda_uxxxx: 0.008, Time: 2.71\n", + "It: 90, Loss: 1.002e+00, loss_u: 1.001e+00, loss_f: 1.830e-03, loss_lambda: 2.689e-08, Lambda_uux: -0.036, Lambda_uxx: -0.012, Lambda_uxxxx: 0.013, Time: 2.70\n", + "It: 100, Loss: 9.948e-01, loss_u: 9.930e-01, loss_f: 1.814e-03, loss_lambda: 2.944e-08, Lambda_uux: -0.051, Lambda_uxx: -0.000, Lambda_uxxxx: 0.015, Time: 2.72\n", + "It: 110, Loss: 9.889e-01, loss_u: 9.870e-01, loss_f: 1.827e-03, loss_lambda: 3.257e-08, Lambda_uux: -0.068, Lambda_uxx: 0.003, Lambda_uxxxx: 0.012, Time: 2.71\n", + "It: 120, Loss: 9.842e-01, loss_u: 9.825e-01, loss_f: 1.755e-03, loss_lambda: 3.524e-08, Lambda_uux: -0.084, Lambda_uxx: -0.004, Lambda_uxxxx: 0.008, Time: 2.70\n", + "It: 130, Loss: 9.779e-01, loss_u: 9.764e-01, loss_f: 1.576e-03, loss_lambda: 3.899e-08, Lambda_uux: -0.099, Lambda_uxx: -0.012, Lambda_uxxxx: 0.008, Time: 2.70\n", + "It: 140, Loss: 9.749e-01, loss_u: 9.733e-01, loss_f: 1.547e-03, loss_lambda: 4.163e-08, Lambda_uux: -0.110, Lambda_uxx: -0.010, Lambda_uxxxx: 0.011, Time: 2.72\n", + "It: 150, Loss: 9.759e-01, loss_u: 9.744e-01, loss_f: 1.534e-03, loss_lambda: 4.491e-08, Lambda_uux: -0.121, Lambda_uxx: -0.008, Lambda_uxxxx: 0.019, Time: 2.72\n", + "It: 160, Loss: 9.755e-01, loss_u: 9.741e-01, loss_f: 1.461e-03, loss_lambda: 4.490e-08, Lambda_uux: -0.133, Lambda_uxx: -0.001, Lambda_uxxxx: 0.010, Time: 2.71\n", + "It: 170, Loss: 9.721e-01, loss_u: 9.707e-01, loss_f: 1.486e-03, loss_lambda: 4.799e-08, Lambda_uux: -0.144, Lambda_uxx: -0.012, Lambda_uxxxx: 0.011, Time: 2.74\n", + "It: 180, Loss: 9.721e-01, loss_u: 9.706e-01, loss_f: 1.440e-03, loss_lambda: 4.967e-08, Lambda_uux: -0.157, Lambda_uxx: -0.019, Lambda_uxxxx: 0.004, Time: 2.73\n", + "It: 190, Loss: 9.735e-01, loss_u: 9.721e-01, loss_f: 1.418e-03, loss_lambda: 5.220e-08, Lambda_uux: -0.169, Lambda_uxx: -0.029, Lambda_uxxxx: 0.005, Time: 2.72\n", + "It: 200, Loss: 9.713e-01, loss_u: 9.699e-01, loss_f: 1.337e-03, loss_lambda: 5.401e-08, Lambda_uux: -0.180, Lambda_uxx: -0.034, Lambda_uxxxx: 0.001, Time: 2.71\n", + "It: 210, Loss: 9.706e-01, loss_u: 9.692e-01, loss_f: 1.425e-03, loss_lambda: 5.662e-08, Lambda_uux: -0.191, Lambda_uxx: -0.041, Lambda_uxxxx: -0.005, Time: 2.71\n", + "It: 220, Loss: 9.686e-01, loss_u: 9.673e-01, loss_f: 1.375e-03, loss_lambda: 5.876e-08, Lambda_uux: -0.202, Lambda_uxx: -0.046, Lambda_uxxxx: -0.006, Time: 2.74\n", + "It: 230, Loss: 9.677e-01, loss_u: 9.663e-01, loss_f: 1.388e-03, loss_lambda: 5.992e-08, Lambda_uux: -0.213, Lambda_uxx: -0.046, Lambda_uxxxx: 0.004, Time: 2.72\n", + "It: 240, Loss: 9.676e-01, loss_u: 9.661e-01, loss_f: 1.486e-03, loss_lambda: 6.205e-08, Lambda_uux: -0.223, Lambda_uxx: -0.037, Lambda_uxxxx: 0.021, Time: 2.72\n", + "It: 250, Loss: 9.684e-01, loss_u: 9.671e-01, loss_f: 1.229e-03, loss_lambda: 6.069e-08, Lambda_uux: -0.233, Lambda_uxx: -0.021, Lambda_uxxxx: 0.013, Time: 2.70\n", + "It: 260, Loss: 9.647e-01, loss_u: 9.633e-01, loss_f: 1.416e-03, loss_lambda: 6.224e-08, Lambda_uux: -0.242, Lambda_uxx: -0.034, Lambda_uxxxx: 0.010, Time: 2.71\n", + "It: 270, Loss: 9.621e-01, loss_u: 9.607e-01, loss_f: 1.396e-03, loss_lambda: 6.493e-08, Lambda_uux: -0.253, Lambda_uxx: -0.052, Lambda_uxxxx: 0.012, Time: 2.72\n", + "It: 280, Loss: 9.598e-01, loss_u: 9.582e-01, loss_f: 1.606e-03, loss_lambda: 6.514e-08, Lambda_uux: -0.261, Lambda_uxx: -0.057, Lambda_uxxxx: 0.003, Time: 2.71\n", + "It: 290, Loss: 9.581e-01, loss_u: 9.563e-01, loss_f: 1.801e-03, loss_lambda: 6.523e-08, Lambda_uux: -0.267, Lambda_uxx: -0.061, Lambda_uxxxx: -0.003, Time: 2.71\n", + "It: 300, Loss: 9.566e-01, loss_u: 9.546e-01, loss_f: 1.953e-03, loss_lambda: 6.513e-08, Lambda_uux: -0.270, Lambda_uxx: -0.065, Lambda_uxxxx: -0.005, Time: 2.73\n", + "It: 310, Loss: 9.619e-01, loss_u: 9.597e-01, loss_f: 2.203e-03, loss_lambda: 6.383e-08, Lambda_uux: -0.269, Lambda_uxx: -0.065, Lambda_uxxxx: -0.003, Time: 2.72\n", + "It: 320, Loss: 9.561e-01, loss_u: 9.539e-01, loss_f: 2.218e-03, loss_lambda: 6.235e-08, Lambda_uux: -0.265, Lambda_uxx: -0.062, Lambda_uxxxx: -0.004, Time: 2.72\n", + "It: 330, Loss: 9.534e-01, loss_u: 9.511e-01, loss_f: 2.237e-03, loss_lambda: 6.176e-08, Lambda_uux: -0.258, Lambda_uxx: -0.062, Lambda_uxxxx: -0.005, Time: 2.71\n", + "It: 340, Loss: 9.550e-01, loss_u: 9.530e-01, loss_f: 1.951e-03, loss_lambda: 6.189e-08, Lambda_uux: -0.247, Lambda_uxx: -0.064, Lambda_uxxxx: -0.004, Time: 2.72\n", + "It: 350, Loss: 9.509e-01, loss_u: 9.488e-01, loss_f: 2.093e-03, loss_lambda: 6.213e-08, Lambda_uux: -0.233, Lambda_uxx: -0.063, Lambda_uxxxx: -0.003, Time: 2.72\n", + "It: 360, Loss: 9.492e-01, loss_u: 9.471e-01, loss_f: 2.096e-03, loss_lambda: 6.202e-08, Lambda_uux: -0.216, Lambda_uxx: -0.058, Lambda_uxxxx: -0.002, Time: 2.70\n", + "It: 370, Loss: 9.456e-01, loss_u: 9.429e-01, loss_f: 2.712e-03, loss_lambda: 6.208e-08, Lambda_uux: -0.200, Lambda_uxx: -0.054, Lambda_uxxxx: -0.003, Time: 2.69\n", + "It: 380, Loss: 9.416e-01, loss_u: 9.392e-01, loss_f: 2.438e-03, loss_lambda: 6.259e-08, Lambda_uux: -0.186, Lambda_uxx: -0.055, Lambda_uxxxx: -0.002, Time: 2.70\n", + "It: 390, Loss: 9.524e-01, loss_u: 9.504e-01, loss_f: 1.969e-03, loss_lambda: 6.241e-08, Lambda_uux: -0.174, Lambda_uxx: -0.051, Lambda_uxxxx: -0.002, Time: 2.72\n", + "It: 400, Loss: 9.411e-01, loss_u: 9.384e-01, loss_f: 2.686e-03, loss_lambda: 6.227e-08, Lambda_uux: -0.162, Lambda_uxx: -0.048, Lambda_uxxxx: -0.002, Time: 2.71\n", + "It: 410, Loss: 9.402e-01, loss_u: 9.374e-01, loss_f: 2.822e-03, loss_lambda: 6.297e-08, Lambda_uux: -0.150, Lambda_uxx: -0.049, Lambda_uxxxx: -0.002, Time: 2.73\n", + "It: 420, Loss: 9.366e-01, loss_u: 9.339e-01, loss_f: 2.694e-03, loss_lambda: 6.415e-08, Lambda_uux: -0.141, Lambda_uxx: -0.050, Lambda_uxxxx: -0.002, Time: 2.69\n", + "It: 430, Loss: 9.362e-01, loss_u: 9.333e-01, loss_f: 2.821e-03, loss_lambda: 6.503e-08, Lambda_uux: -0.134, Lambda_uxx: -0.049, Lambda_uxxxx: -0.003, Time: 2.69\n", + "It: 440, Loss: 9.325e-01, loss_u: 9.288e-01, loss_f: 3.690e-03, loss_lambda: 6.555e-08, Lambda_uux: -0.129, Lambda_uxx: -0.047, Lambda_uxxxx: -0.003, Time: 2.72\n", + "It: 450, Loss: 9.346e-01, loss_u: 9.304e-01, loss_f: 4.119e-03, loss_lambda: 6.614e-08, Lambda_uux: -0.127, Lambda_uxx: -0.046, Lambda_uxxxx: -0.003, Time: 2.70\n", + "It: 460, Loss: 9.408e-01, loss_u: 9.385e-01, loss_f: 2.311e-03, loss_lambda: 6.689e-08, Lambda_uux: -0.129, Lambda_uxx: -0.046, Lambda_uxxxx: -0.004, Time: 2.72\n", + "It: 470, Loss: 9.487e-01, loss_u: 9.458e-01, loss_f: 2.874e-03, loss_lambda: 6.778e-08, Lambda_uux: -0.132, Lambda_uxx: -0.047, Lambda_uxxxx: -0.005, Time: 2.72\n", + "It: 480, Loss: 9.364e-01, loss_u: 9.345e-01, loss_f: 1.819e-03, loss_lambda: 6.915e-08, Lambda_uux: -0.133, Lambda_uxx: -0.047, Lambda_uxxxx: -0.007, Time: 2.72\n", + "It: 490, Loss: 9.298e-01, loss_u: 9.268e-01, loss_f: 3.022e-03, loss_lambda: 7.036e-08, Lambda_uux: -0.137, Lambda_uxx: -0.046, Lambda_uxxxx: -0.006, Time: 2.71\n", + "It: 500, Loss: 9.227e-01, loss_u: 9.185e-01, loss_f: 4.195e-03, loss_lambda: 7.049e-08, Lambda_uux: -0.153, Lambda_uxx: -0.043, Lambda_uxxxx: -0.001, Time: 2.72\n", + "It: 510, Loss: 9.152e-01, loss_u: 9.109e-01, loss_f: 4.319e-03, loss_lambda: 6.961e-08, Lambda_uux: -0.180, Lambda_uxx: -0.038, Lambda_uxxxx: -0.001, Time: 2.71\n", + "It: 520, Loss: 9.371e-01, loss_u: 9.329e-01, loss_f: 4.194e-03, loss_lambda: 6.900e-08, Lambda_uux: -0.214, Lambda_uxx: -0.043, Lambda_uxxxx: -0.002, Time: 2.71\n", + "It: 530, Loss: 9.078e-01, loss_u: 9.035e-01, loss_f: 4.278e-03, loss_lambda: 6.863e-08, Lambda_uux: -0.246, Lambda_uxx: -0.045, Lambda_uxxxx: -0.001, Time: 2.71\n", + "It: 540, Loss: 9.105e-01, loss_u: 9.054e-01, loss_f: 5.100e-03, loss_lambda: 6.960e-08, Lambda_uux: -0.275, Lambda_uxx: -0.042, Lambda_uxxxx: -0.002, Time: 2.70\n", + "It: 550, Loss: 9.005e-01, loss_u: 8.956e-01, loss_f: 4.898e-03, loss_lambda: 7.213e-08, Lambda_uux: -0.302, Lambda_uxx: -0.047, Lambda_uxxxx: -0.001, Time: 2.70\n", + "It: 560, Loss: 9.124e-01, loss_u: 9.073e-01, loss_f: 5.051e-03, loss_lambda: 7.482e-08, Lambda_uux: -0.328, Lambda_uxx: -0.054, Lambda_uxxxx: -0.001, Time: 2.72\n", + "It: 570, Loss: 9.070e-01, loss_u: 9.026e-01, loss_f: 4.311e-03, loss_lambda: 7.724e-08, Lambda_uux: -0.350, Lambda_uxx: -0.060, Lambda_uxxxx: -0.002, Time: 2.72\n", + "It: 580, Loss: 8.993e-01, loss_u: 8.942e-01, loss_f: 5.032e-03, loss_lambda: 7.835e-08, Lambda_uux: -0.371, Lambda_uxx: -0.057, Lambda_uxxxx: -0.001, Time: 2.72\n", + "It: 590, Loss: 8.963e-01, loss_u: 8.913e-01, loss_f: 5.062e-03, loss_lambda: 7.946e-08, Lambda_uux: -0.392, Lambda_uxx: -0.055, Lambda_uxxxx: -0.001, Time: 2.71\n", + "It: 600, Loss: 8.942e-01, loss_u: 8.889e-01, loss_f: 5.303e-03, loss_lambda: 8.157e-08, Lambda_uux: -0.414, Lambda_uxx: -0.060, Lambda_uxxxx: -0.001, Time: 2.72\n", + "It: 610, Loss: 9.091e-01, loss_u: 9.040e-01, loss_f: 5.119e-03, loss_lambda: 8.314e-08, Lambda_uux: -0.434, Lambda_uxx: -0.059, Lambda_uxxxx: -0.001, Time: 2.71\n", + "It: 620, Loss: 8.914e-01, loss_u: 8.864e-01, loss_f: 4.988e-03, loss_lambda: 8.435e-08, Lambda_uux: -0.453, Lambda_uxx: -0.060, Lambda_uxxxx: -0.001, Time: 2.71\n", + "It: 630, Loss: 8.900e-01, loss_u: 8.848e-01, loss_f: 5.195e-03, loss_lambda: 8.533e-08, Lambda_uux: -0.469, Lambda_uxx: -0.061, Lambda_uxxxx: -0.001, Time: 2.72\n", + "It: 640, Loss: 8.898e-01, loss_u: 8.842e-01, loss_f: 5.603e-03, loss_lambda: 8.592e-08, Lambda_uux: -0.486, Lambda_uxx: -0.059, Lambda_uxxxx: -0.001, Time: 2.71\n", + "It: 650, Loss: 8.925e-01, loss_u: 8.867e-01, loss_f: 5.759e-03, loss_lambda: 8.690e-08, Lambda_uux: -0.501, Lambda_uxx: -0.059, Lambda_uxxxx: -0.001, Time: 2.71\n", + "It: 660, Loss: 8.884e-01, loss_u: 8.829e-01, loss_f: 5.522e-03, loss_lambda: 8.766e-08, Lambda_uux: -0.515, Lambda_uxx: -0.059, Lambda_uxxxx: -0.001, Time: 2.72\n", + "It: 670, Loss: 8.842e-01, loss_u: 8.785e-01, loss_f: 5.737e-03, loss_lambda: 8.794e-08, Lambda_uux: -0.528, Lambda_uxx: -0.057, Lambda_uxxxx: -0.001, Time: 2.74\n", + "It: 680, Loss: 8.909e-01, loss_u: 8.849e-01, loss_f: 6.057e-03, loss_lambda: 8.817e-08, Lambda_uux: -0.540, Lambda_uxx: -0.055, Lambda_uxxxx: -0.001, Time: 2.69\n", + "It: 690, Loss: 8.858e-01, loss_u: 8.798e-01, loss_f: 6.055e-03, loss_lambda: 8.863e-08, Lambda_uux: -0.549, Lambda_uxx: -0.056, Lambda_uxxxx: -0.001, Time: 2.72\n" + ] + } + ], + "source": [ + "for epoch, lr in zip(epochs, learning_rate):\n", + " train(model_pretrain, int(epoch), lr)\n", + "ms.save_checkpoint(model_pretrain.dnn, f'model/{second_path}/model.ckpt')" + ] + }, + { + "cell_type": "markdown", + "id": "60d9233f-3748-4481-9d91-0c6c4ff42d26", + "metadata": {}, + "source": [ + "保存最后一次训练的可学习参数用于方程发现" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b2ba9195-6d20-46ef-8bb4-f77e00914aae", + "metadata": {}, + "outputs": [], + "source": [ + "lambda_uux_value = model_pretrain.lambda_uux.numpy()\n", + "lambda_u_value = model_pretrain.lambda_u.numpy()\n", + "np.save(f'Loss-Coe/{second_path}/lambda_uux_value', lambda_uux_value)\n", + "np.save(f'Loss-Coe/{second_path}/lambda_u_value', lambda_u_value)" + ] + }, + { + "cell_type": "markdown", + "id": "a5f276c8-9b5a-4f01-a76e-90e4c5addff9", + "metadata": {}, + "source": [ + "训练结束后直接进行方程发现可能会超出显存，因此需要根据计算机显存大小判断是否需要重新加载模型进行方程发现" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1b672fa0-3903-4ac7-9395-211895cfaeba", + "metadata": {}, + "outputs": [], + "source": [ + "if load_params:\n", + " lambda_u_value = np.load(f'Loss-Coe/{second_path}/lambda_u_value.npy')\n", + " lambda_uux_value = np.load(f'Loss-Coe/{second_path}/lambda_uux_value.npy')\n", + " model_ICCO = InvarianceConstrainedNN_STRdige(X_u_train, u_train, X_f_train, network_size, BatchNo, lambda_u_value, lambda_uux_value, load_params, second_path, msfloat_type)\n", + "else:\n", + " model_ICCO = InvarianceConstrainedNN_STRdige(X_u_train, u_train, X_f_train, network_size, BatchNo, lambda_u_value, lambda_uux_value, load_params, second_path, msfloat_type)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1b8887fa-9550-4873-a5cf-edd2a49b76ff", + "metadata": { + "scrolled": true + }, + "outputs": [], + "source": [ + "lambda_u_STRidge = model_ICCO.call_trainstridge(lam, d_tol)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "fdb0acd9-8890-4bb9-bde1-77cf4e9b17f3", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "u_t = (0.973947)uu_x\n", + " (-0.967219)u_xx\n", + " + (-0.967183)u_xxxx\n", + " \n" + ] + } + ], + "source": [ + "# GPU results\n", + "print_pde(lambda_uux_value, lambda_u_STRidge, description_ks, ut='u_t')" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.19" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/MindFlow/applications/research/ICNet/README.md b/MindFlow/applications/research/ICNet/README.md new file mode 100644 index 0000000000000000000000000000000000000000..5909b3b0a1412e8221abc3ce2bd1080fa1becc4f --- /dev/null +++ b/MindFlow/applications/research/ICNet/README.md @@ -0,0 +1,82 @@ +# ICNet: Invariance Constrained Discovery for Partial Differential Equations + +## Background 1 + +The physical laws described by partial differential equations are widely present in the natural environment. The calculation and simulation of physical systems rely on accurate basic equations and models. The traditional method of deriving control equations is mainly based on first principles, such as the Navier-Stokes equations based on momentum conservation. The difficulty of traditional methods lies in the fact that models and equations of complex dynamics are often difficult to derive, such as multiphase flow, neuroscience, and biological science. In the era of big data, mining control equations from data through artificial intelligence methods has become a new research idea. The existing data-driven method of discovering equations still has certain limitations. At present, there is a lack of guiding principles when constructing candidates for a complete library, and it is impossible to ensure that the discovered equations meet basic physical requirements. At the same time, when dealing with complex multidimensional systems, the candidate library is too large, and it is difficult to discover simple and accurate equations. Considering that basic physical requirements (invariance, conservation, etc.) are the cornerstones of many physical problems, it is necessary to study how to impose physical constraints in discovering equations. + +## Model framework + +The model framework is as shown in the following figure: + +![ICNet](images/ICNet.png) + +In the figure: A. Schematic diagram of the derivation process of embedding invariance constraints into the framework of partial differential equation discovery; B. The neural network module of partial differential equation discovery with invariance constraints uses neural network automatic differentiation to obtain the partial derivatives required to construct the invariance candidate function. The loss function includes data loss, invariance loss, and regularization loss for enhancing sparsity. + +## QuickStart + +Dataset download link: [ICNet/dataset](https://download-mindspore.osinfra.cn/mindscience/mindflow/dataset/applications/research/ICNet/)., Save the dataset to `./dataset`. + +The case provides two training methods + +- Run Option 1: Call `train_GI_1D_KS.py` from command line + + ```python + # Call `train.py` from command line + python train.py --device_target GPU --device_id 0 --config_file_path ./config/ICNet_KS.yaml + + ``` + + `--config_path` indicates the path of the parameter file. Default path "./config/config.yaml". + + In the "./config/config.yaml" parameter file: + + 'lambda_GP' represents the gradient penalty coefficient, with a default value of 10; + + 'critic_model_lr' represents the learning rate of discriminator, with a default value of 0.00025; + + 'gan_model_lr' represents the learning rate of generator, with a default value of 0.004； + + 'case' represents the case name; + + 'device' representsthe type of computing platform used, with options of 'CPU', 'GPU', or 'Ascend', with a default value of 'GPU'; + + 'device_id' represents the index of NPU or GPU. Default 0; + + 'network_size' represents the size of network； + + 'learning_rate' represents the learning rate； + + 'epochs' represents number of training iterations； + +- Run Option 2: Run Jupyter Notebook + + You can use [chinese](./ICNet_CN.ipynb) or [English](./ICNet.ipynb) Jupyter Notebook to run the training and evaluation code line-by-line. + +## Case description + +The case run by the Jupyter Notebook named ICNet is the one-dimensional Kuramoto–Sivashinsky equation with Galileo invariance constraints. In addition to providing the running code and data of this equation, this open source also provides the code and data of the two-dimensional Burgers equation (Galileo invariance), the two-dimensional univariate Klein-Gordon equation (Lorentz invariance), and the two-dimensional coupled variable Klein-Gordon equation (Lorentz invariance). You can call the `train_GI_2D_Burgers.py`, `train_LI_2D_SKG.py` and `train_LI_2D_CKG.py` scripts in the command line to run them directly. + +## Performance + +| Parameters | GPU | NPU | +|:-------------------------:|:-----------------------:|:------------------:| +| hardware | NVIDIA 1080Ti(memory 11G) | Ascend(memory 32G) | +| MindSpore version | 2.2.14 | 2.2.14 | +| data_size | 12800 | 12800 | +| batch_size | 1 | 1 | +| epochs | 60w | 60w | +| optimizer | Adam | Adam | +| total loss (MSE) | 1.8e-3 | 9.4e-4 | +| data loss (MSE) | 1.3e-3 | 3.7e-4 | +| invariance loss (MSE) | 5.5e-4 | 5.7e-4 | +| regularization loss (MSE)| 1.9e-7 | 1.8e-7 | +| speed(s/step) | 0.27 | 0.041 | + +## Contributor + +gitee id: [chenchao2024](https://gitee.com/chenchao2024) +email: chenchao@isrc.iscas.ac.cn + +## Reference + +chen c, Li H, Jin X. An invariance constrained deep learning network for partial differential equation discovery[J]. Physics of Fluids, 2043, 65: 471202. https://doi.org10.1063/5.02026339 diff --git a/MindFlow/applications/research/ICNet/README_CN.md b/MindFlow/applications/research/ICNet/README_CN.md new file mode 100644 index 0000000000000000000000000000000000000000..4f72ed0f351e3befb222a08ec4a3d325c024279f --- /dev/null +++ b/MindFlow/applications/research/ICNet/README_CN.md @@ -0,0 +1,80 @@ +# ICNet不变性约束偏微分控制方程发现 + +## 背景介绍 + +由偏微分方程描述的物理规律广泛存在于自然环境之中，物理系统的计算与模拟依赖于准确的基本方程和模型，传统方法推导控制方程主要基于第一性原理，例如Navier-Stokes方程基于动量守恒，传统方法难点在于复杂动力学的模型与方程常常难以推导，例如多相流、神经科学以及生物科学等，在大数据时代，通过人工智能的方法从数据中挖掘控制方程成为一种新的研究思路。已有的数据驱动发现方程的方法依然存在一定的局限性，目前构建过完备库的候选项时缺乏指导原则，无法保证发现的方程满足基本的物理要求，同时在处理复杂多维系统时候选库过大，而难以发现出简约准确的方程。考虑到基本的物理要求（不变性，守恒性等）是很多物理问题出发的基石，因此有必要研究如何在发现方程中施加物理约束。 + +## 模型框架 + +模型框架如下图所示： + +![ICNet](images/ICNet.png) + +图中： + +A. 嵌入不变性约束至发现偏微分方程框架中的推导过程示意图； + +B. 不变性约束发现偏微分方程的神经网络模块，利用神经网络自动微分求出构建不变性候选函数所需要的偏导数，损失函数包括数据损失Data loss，不变性损失Invariance loss以及增强稀疏性的正则化损失Regularization loss。 + +## 快速开始 + +数据集下载地址：[ICNet/dataset](https://download-mindspore.osinfra.cn/mindscience/mindflow/dataset/applications/research/ICNet/). 将数据集保存在`./dataset`路径下。 + +案例提供两种训练方式 + +- 训练方式一：在命令行中调用`train_GI_1D_KS.py`脚本 + + ```python + # 在命令行调用train.py进行训练示例 + python train.py --device_target GPU --device_id 0 --config_file_path ./config/ICNet_KS.yaml + + ``` + + `--config_path`表示配置文件的路径，默认值为 "./config/config.yaml"。 + + 在 "./config/config.yaml" 配置文件中： + 'case' 表示案例名称； + + 'device' 表示使用的计算平台类型，可以选择 'CPU'、'GPU' 或 'Ascend'，默认值 'GPU'； + + 'device_id' 表示后端平台端口号，默认值为 0； + + 'network_size' 表示网络大小； + + 'learning_rate' 表示学习率； + + 'epochs' 表示训练迭代次数； + +- 训练方式二：运行Jupyter Notebook + + 您可以使用[中文版](./ICNet_CN.ipynb) 和[英文版](./ICNet.ipynb) Jupyter Notebook逐行运行训练和验证代码。 + +## 案例说明 + +其中命名为ICNet的Jupyter Notebook 运行的案例为施加伽利略不变性约束的一维Kuramoto–Sivashinsky 方程，本次开源除了提供该方程的运行代码和数据以外，还分别提供了二维Burgers方程（伽利略不变性）、二维单变量 Klein-Gordon 方程（洛伦兹不变性）以及二维耦合变量的 Klein-Gordon 方程（洛伦兹不变性）的代码和数据，可以在命令行中调用`train_GI_2D_Burgers.py`、`train_LI_2D_SKG.py`以及`train_LI_2D_CKG.py` 脚本即可直接运行。 + +## 性能 + +| 参数 | GPU | NPU | +|:-------------------:|:-----------------------:|:------------------:| +| 硬件 | NVIDIA 1080Ti(memory 11G) | Ascend(memory 32G) | +| MindSpore版本 | 2.2.14 | 2.2.14 | +| 数据大小 | 12800 | 12800 | +| batch大小 | 1 | 1 | +| 训练步数 | 60w | 60w | +| 优化器 | Adam | Adam | +| total loss 训练精度(MSE) | 1.8e-3 | 9.4e-4 | +| data loss 测试精度(MSE) | 1.3e-3 | 3.7e-4 | +| invariance loss 训练精度(MSE) | 5.5e-4 | 5.7e-4 | +| regularization loss 测试精度(MSE) | 1.9e-7 | 1.8e-7 | +| 性能(s/epoch) | 0.27 | 0.041 | + +## 贡献者 + +gitee id: [chenchao2024](https://gitee.com/chenchao2024) +email: chenchao@isrc.iscas.ac.cn + +## 参考文献 + +chen c, Li H, Jin X. An invariance constrained deep learning network +for partial differential equation discovery[J]. Physics of Fluids, 2043, 65: 471202. https://doi.org10.1063/5.02026339 diff --git a/MindFlow/applications/research/ICNet/config/ICNet_Burgers.yaml b/MindFlow/applications/research/ICNet/config/ICNet_Burgers.yaml new file mode 100644 index 0000000000000000000000000000000000000000..e961f0e1bf97330a4a3ff65b89151d542dd90575 --- /dev/null +++ b/MindFlow/applications/research/ICNet/config/ICNet_Burgers.yaml @@ -0,0 +1,21 @@ +params: + model_name: 'ICNet' + case: 'Two-dimensional Burgers equation' + data_name: 'Burgers2D.mat' + second_path: 'pretrain' + device: 'GPU' + init_steps: 30 + stop_steps: 150 + time_steps: 40 + device_id: 3 + load_params: 'True' + network_size: [3, 60, 60, 60, 60, 60, 60, 60, 60, 2] + description_burgers_u: ['uu_x','vu_y','1','u_x', 'u_xx', 'u_y', 'u_yy', 'u_xy', 'v_x', 'v_xx', 'v_y', 'v_yy', 'v_xy'] + description_burgers_v: ['uv_x','vv_y','1','u_x', 'u_xx', 'u_y', 'u_yy', 'u_xy', 'v_x', 'v_xx', 'v_y', 'v_yy', 'v_xy'] + learning_rate: [0.001, 0.0005, 1.0e-04, 1.0e-05] + epochs: [3e5, 3e5, 1e4, 1e4] + BatchNo: 1 + lam: 10**-5 + d_tol: 5.0 + second_path1: 'pretrain' + diff --git a/MindFlow/applications/research/ICNet/config/ICNet_CKG.yaml b/MindFlow/applications/research/ICNet/config/ICNet_CKG.yaml new file mode 100644 index 0000000000000000000000000000000000000000..fcf6b12cb2d599f3d178a81afb0457dce639603d --- /dev/null +++ b/MindFlow/applications/research/ICNet/config/ICNet_CKG.yaml @@ -0,0 +1,21 @@ +params: + model_name: 'ICNet' + case: 'Coupled Klein-Gordon equation' + data_name: 'Coupled-Klein-Gordon.mat' + second_path: 'pretrain' + device: 'GPU' + init_steps: 30 + stop_steps: 200 + time_steps: 40 + device_id: 3 + load_params: 'True' + second_path1: 'pretrain' + network_size: [3, 60, 60, 60, 60, 60, 60, 60, 60, 2] + description_CKG_u: ['u_xx','u_yy','1','u', 'u*u', 'u*u*u', 'u*v', 'u*v*v', 'v', 'v*v', 'v*v*v', 'v*u*u'] + description_CKG_v: ['v_xx','v_yy','1','u', 'u*u', 'u*u*u', 'u*v', 'u*v*v', 'v', 'v*v', 'v*v*v', 'v*u*u'] + learning_rate: [0.001, 0.0005, 1.0e-04, 1.0e-05] + epochs: [3e5, 3e5, 1e4, 1e4] + BatchNo: 1 + lam: 1e-5 + d_tol: 1.0 + diff --git a/MindFlow/applications/research/ICNet/config/ICNet_KS.yaml b/MindFlow/applications/research/ICNet/config/ICNet_KS.yaml new file mode 100644 index 0000000000000000000000000000000000000000..0208bc65a1456715690e0861b36ce6334f548d9f --- /dev/null +++ b/MindFlow/applications/research/ICNet/config/ICNet_KS.yaml @@ -0,0 +1,19 @@ +params: + model_name: 'ICNet' + data_name: 'KS.mat' + init_steps: 0 + stop_steps: 150 + time_steps: 50 + case: 'Kuramoto-Sivashinsky equation' + device: 'GPU' + device_id: 3 + load_params: 'True' + second_path: 'pretrain' + network_size: [2, 40, 40, 40, 40, 40, 40, 40, 40, 1] + description_ks: ['uu_x','1','u_x','u_xx', 'u_xxx','u_xxxx'] + learning_rate: [0.001, 0.0005, 1.0e-04, 1.0e-05] + epochs: [3e5, 3e5, 1e4, 1e4] + BatchNo: 1 + lam: 1e-5 + d_tol: 1.0 + diff --git a/MindFlow/applications/research/ICNet/config/ICNet_SKG.yaml b/MindFlow/applications/research/ICNet/config/ICNet_SKG.yaml new file mode 100644 index 0000000000000000000000000000000000000000..45c78593f7a038d4334b472e45e96e2b6336bbe4 --- /dev/null +++ b/MindFlow/applications/research/ICNet/config/ICNet_SKG.yaml @@ -0,0 +1,20 @@ +params: + model_name: 'ICNet' + case: 'Single Klein-Gordon equation' + data_name: 'Single-Klein-Gordon.mat' + second_path: 'pretrain' + device: 'GPU' + init_steps: 50 + stop_steps: 230 + time_steps: 50 + device_id: 3 + load_params: 'True' + second_path1: 'pretrain' + network_size: [3, 30, 30, 30, 30, 30, 30, 30, 30, 1] + description_SKG: ['u_xx','u_yy','1','u','u**2', 'u**3'] + learning_rate: [0.001, 0.0005, 1.0e-04, 1.0e-05] + epochs: [3e5, 3e5, 1e4, 1e4] + BatchNo: 1 + lam: 1e-5 + d_tol: 1.0 + diff --git a/MindFlow/applications/research/ICNet/images/ICNet.png b/MindFlow/applications/research/ICNet/images/ICNet.png new file mode 100644 index 0000000000000000000000000000000000000000..f390d42b89642b4347a2dd628c8573dab5a82765 Binary files /dev/null and b/MindFlow/applications/research/ICNet/images/ICNet.png differ diff --git a/MindFlow/applications/research/ICNet/src/__init__.py b/MindFlow/applications/research/ICNet/src/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..14331ea3db82dc710550f88802dc5c8dad5f8a81 --- /dev/null +++ b/MindFlow/applications/research/ICNet/src/__init__.py @@ -0,0 +1,14 @@ +# Copyright 2023 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================ diff --git a/MindFlow/applications/research/ICNet/src/datasets.py b/MindFlow/applications/research/ICNet/src/datasets.py new file mode 100644 index 0000000000000000000000000000000000000000..883fe190ca7186662b0b687bfb215f9e7a65bd23 --- /dev/null +++ b/MindFlow/applications/research/ICNet/src/datasets.py @@ -0,0 +1,89 @@ +# Copyright 2023 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================ +# pylint: disable=C0103 +"""ICNet dataset""" +import numpy as np + +import scipy.io +from pyDOE import lhs + +def read_training_data(args): + """prepare data""" + data_name = args.data_name + init_steps = args.init_steps + stop_steps = args.stop_steps + time_steps = args.time_steps + + data = scipy.io.loadmat(data_name) + + t = np.real(data['t_2'][:, init_steps:stop_steps].flatten()[:, None]) + x = np.real(data['x_4'].flatten()[:, None]) + exact = np.real(data['u_4_2'][:, init_steps:stop_steps]).T + + X, T = np.meshgrid(x, t) + + X_star = np.hstack((X.flatten()[:, None], T.flatten()[:, None])) + + # Domain bounds + lb = X_star.min(0) + ub = X_star.max(0) + + # Sample the datapoints at spatial domain + N_u_s = 256 + idx_s = np.random.choice(x.shape[0], N_u_s, replace=False) + + # Sample the datapoints at temporal domain + N_u_t = time_steps + idx_t = np.random.choice(t.shape[0], N_u_t, replace=False) + idx_t = idx_t.astype(np.int32) + + X1 = X[:, idx_s] + X2 = X1[idx_t, :] + T1 = T[:, idx_s] + T2 = T1[idx_t, :] + exact1 = exact[:, idx_s] + exact2 = exact1[idx_t, :] + + X_u_meas = np.hstack((X2.flatten()[:, None], T2.flatten()[:, None])) + u_meas = exact2.flatten()[:, None] + + # Prepare the training datasets + N_u_train = int(N_u_s*N_u_t) + idx_train = np.random.choice(X_u_meas.shape[0], N_u_train, replace=False) + X_u_train = X_u_meas[idx_train, :] + u_train = u_meas[idx_train, :] + + # Collocation points + N_f = 20000 + X_f_train = lb + (ub-lb)*lhs(2, N_f) + X_f_train = np.vstack((X_f_train, X_u_train)) + # X_f_train = X_u_train + + + # Add Noise + noise = 0.0 + u_train = u_train + noise*np.std(u_train)*np.random.randn(u_train.shape[0], u_train.shape[1]) + return X_u_train, u_train, X_f_train + +def print_pde(lambda_uux, w, rhs_description, ut='u_t'): + pde = ut + '=' + "(%05f)" % (lambda_uux.real) + rhs_description[0] + "\n " + first = True + for i in range(len(w)): + if w[i] != 0: + if not first: + pde = pde + ' + ' + pde = pde + "(%05f)" % (w[i].real) + rhs_description[i+1] + "\n " + first = False + print(pde) diff --git a/MindFlow/applications/research/ICNet/src/datasets_CKG.py b/MindFlow/applications/research/ICNet/src/datasets_CKG.py new file mode 100644 index 0000000000000000000000000000000000000000..7b7621d1d95ac35eb8e6240f506b648d798006b7 --- /dev/null +++ b/MindFlow/applications/research/ICNet/src/datasets_CKG.py @@ -0,0 +1,105 @@ +# Copyright 2023 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================ +# pylint: disable=C0103 +"""ICNet dataset""" +import numpy as np + +import scipy.io +from pyDOE import lhs + +def read_training_data(args): + """prepare data""" + + data_name = args.data_name + init_steps = args.init_steps + stop_steps = args.stop_steps + time_steps = args.time_steps + + data = scipy.io.loadmat(data_name) + + t = np.real(data['t'][init_steps:stop_steps, :].flatten()[:, None]) + x = np.real(data['x'].flatten()[:, None]) + y = np.real(data['y'].flatten()[:, None]) + exact_u = data['U'][:, :, init_steps:stop_steps] + exact_v = data['V'][:, :, init_steps:stop_steps] + + X, Y, T = np.meshgrid(x, y, t) + + X_star = np.hstack((X.flatten()[:, None], Y.flatten()[:, None], T.flatten()[:, None])) + + # Domain bounds + lb = X_star.min(0) + ub = X_star.max(0) + + # Sample the datapoints at spatial domain + N_uv_s = 3000 + idx = np.random.choice(X.shape[0]*X.shape[1], N_uv_s, replace=False) + idx_remainder = idx%(X.shape[0]) + idx_s_y = np.floor(idx/(X.shape[0])) + idx_s_y = idx_s_y.astype(np.int32) + idx_idx_remainder = np.where(idx_remainder == 0)[0] + idx_remainder[idx_idx_remainder] = X.shape[0] + idx_s_x = idx_remainder-1 + + # Sample the datapoints at temporal domain + N_t_s = time_steps + idx_t = np.concatenate([np.random.choice(X.shape[2], N_t_s, replace=False)]) + idx_t = idx_t.astype(np.int32) + + X1 = X[idx_s_x, idx_s_y, :] + X2 = X1[:, idx_t] + Y1 = Y[idx_s_x, idx_s_y, :] + Y2 = Y1[:, idx_t] + T1 = T[idx_s_x, idx_s_y, :] + T2 = T1[:, idx_t] + exact_u1 = exact_u[idx_s_x, idx_s_y, :] + exact_u2 = exact_u1[:, idx_t] + exact_v1 = exact_v[idx_s_x, idx_s_y, :] + exact_v2 = exact_v1[:, idx_t] + + X_star_meas = np.hstack((X2.flatten()[:, None], Y2.flatten()[:, None], T2.flatten()[:, None])) + u_star_meas = exact_u2.flatten()[:, None] + v_star_meas = exact_v2.flatten()[:, None] + + # Prepare the training datasets + N_u_train = int(N_uv_s*N_t_s) + idx_train = np.random.choice(X_star_meas.shape[0], N_u_train, replace=False) + X_star_train = X_star_meas[idx_train, :] + u_star_train = u_star_meas[idx_train, :] + v_star_train = v_star_meas[idx_train, :] + + # Collocation points + N_f = 20000 + X_f = lb + (ub-lb)*lhs(3, N_f) + X_f = np.vstack((X_f, X_star_train)) + + # Add Noise + noise = 0.0 + u_star_train = u_star_train + noise*np.std(u_star_train)*np.random.randn(u_star_train.shape[0], + u_star_train.shape[1]) + v_star_train = v_star_train + noise*np.std(v_star_train)*np.random.randn(v_star_train.shape[0], + v_star_train.shape[1]) + return X_star_train, u_star_train, v_star_train, X_f + +def print_pde(lambda_uxx, lambda_uyy, w, rhs_description, ut='u_t'): + pde = ut + ' = ' + "(%05f)" % (lambda_uxx.real) + rhs_description[0] + "\n" + "(%05f)" % (lambda_uyy.real) + rhs_description[1] + "\n" + first = True + for i in range(len(w)): + if w[i] != 0: + if not first: + pde = pde + ' + ' + pde = pde + "(%05f)" % (w[i].real) + rhs_description[i+2] + "\n " + first = False + print(pde) diff --git a/MindFlow/applications/research/ICNet/src/datasets_SKG.py b/MindFlow/applications/research/ICNet/src/datasets_SKG.py new file mode 100644 index 0000000000000000000000000000000000000000..ca0b09173220244d9ae4d9c699aefaac82374dd8 --- /dev/null +++ b/MindFlow/applications/research/ICNet/src/datasets_SKG.py @@ -0,0 +1,98 @@ +# Copyright 2023 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================ +# pylint: disable=C0103 +"""ICNet dataset""" +import numpy as np + +import scipy.io +from pyDOE import lhs + +def read_training_data(args): + """prepare data""" + data_name = args.data_name + init_steps = args.init_steps + stop_steps = args.stop_steps + time_steps = args.time_steps + + data = scipy.io.loadmat(data_name) + + t = np.real(data['t'][init_steps:stop_steps, :].flatten()[:, None]) + x = np.real(data['x'].flatten()[:, None]) + y = np.real(data['y'].flatten()[:, None]) + exact_u = data['U'][:, :, init_steps:stop_steps] + + X, Y, T = np.meshgrid(x, y, t) + + X_star = np.hstack((X.flatten()[:, None], Y.flatten()[:, None], T.flatten()[:, None])) + + # Domain bounds + lb = X_star.min(0) + ub = X_star.max(0) + + # Sample the datapoints at spatial domain + N_uv_s = 3000 + idx = np.random.choice(X.shape[0]*X.shape[1], N_uv_s, replace=False) + idx_remainder = idx%(X.shape[0]) + idx_s_y = np.floor(idx/(X.shape[0])) + idx_s_y = idx_s_y.astype(np.int32) + idx_idx_remainder = np.where(idx_remainder == 0)[0] + idx_remainder[idx_idx_remainder] = X.shape[0] + idx_s_x = idx_remainder-1 + + # Sample the datapoints at temporal domain + N_t_s = time_steps + idx_t = np.concatenate([np.random.choice(X.shape[2], N_t_s, replace=False)]) + idx_t = idx_t.astype(np.int32) + + X1 = X[idx_s_x, idx_s_y, :] + X2 = X1[:, idx_t] + Y1 = Y[idx_s_x, idx_s_y, :] + Y2 = Y1[:, idx_t] + T1 = T[idx_s_x, idx_s_y, :] + T2 = T1[:, idx_t] + exact_u1 = exact_u[idx_s_x, idx_s_y, :] + exact_u2 = exact_u1[:, idx_t] + + X_star_meas = np.hstack((X2.flatten()[:, None], Y2.flatten()[:, None], T2.flatten()[:, None])) + u_star_meas = exact_u2.flatten()[:, None] + + # Prepare the training datasets + N_u_train = int(N_uv_s*N_t_s) + idx_train = np.random.choice(X_star_meas.shape[0], N_u_train, replace=False) + X_star_train = X_star_meas[idx_train, :] + u_star_train = u_star_meas[idx_train, :] + + # Collocation points + N_f = 20000 + X_f = lb + (ub-lb)*lhs(3, N_f) + X_f = np.vstack((X_f, X_star_train)) + + + # Add Noise + noise = 0.0 + u_star_train = u_star_train + noise*np.std(u_star_train)*np.random.randn(u_star_train.shape[0], + u_star_train.shape[1]) + return X_star_train, u_star_train, X_f + +def print_pde(lambda_uxx, lambda_uyy, w, rhs_description, ut='u_t'): + pde = ut + ' = ' + "(%05f)" % (lambda_uxx.real) + rhs_description[0] + "\n" + "(%05f)" % (lambda_uyy.real) + rhs_description[1] + "\n" + first = True + for i in range(len(w)): + if w[i] != 0: + if not first: + pde = pde + ' + ' + pde = pde + "(%05f)" % (w[i].real) + rhs_description[i+2] + "\n " + first = False + print(pde) diff --git a/MindFlow/applications/research/ICNet/src/datasets_burgers.py b/MindFlow/applications/research/ICNet/src/datasets_burgers.py new file mode 100644 index 0000000000000000000000000000000000000000..24341774dbde2cd13fe28b248c61b8ebbaa1cc50 --- /dev/null +++ b/MindFlow/applications/research/ICNet/src/datasets_burgers.py @@ -0,0 +1,107 @@ +# Copyright 2023 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================ +# pylint: disable=C0103 +"""ICNet dataset""" +import numpy as np + +import scipy.io +from pyDOE import lhs + +def read_training_data(args): + """prepare data""" + + data_name = args.data_name + init_steps = args.init_steps + stop_steps = args.stop_steps + time_steps = args.time_steps + + data = scipy.io.loadmat(data_name) + + t = np.real(data['t'][init_steps:stop_steps, :].flatten()[:, None]) + x = np.real(data['x'].flatten()[:, None]) + y = np.real(data['y'].flatten()[:, None]) + exact_u = data['U'][:, :, init_steps:stop_steps] + exact_v = data['V'][:, :, init_steps:stop_steps] + + X, Y, T = np.meshgrid(x, y, t) + + X_star = np.hstack((X.flatten()[:, None], Y.flatten()[:, None], T.flatten()[:, None])) + + # Domain bounds + lb = X_star.min(0) + ub = X_star.max(0) + + # Sample the datapoints at spatial domain + N_uv_s = 3000 + idx = np.random.choice(X.shape[0]*X.shape[1], N_uv_s, replace=False) + idx_remainder = idx%(X.shape[0]) + idx_s_y = np.floor(idx/(X.shape[0])) + idx_s_y = idx_s_y.astype(np.int32) + idx_idx_remainder = np.where(idx_remainder == 0)[0] + idx_remainder[idx_idx_remainder] = X.shape[0] + idx_s_x = idx_remainder-1 + + # Sample the datapoints at temporal domain + N_t_s = time_steps + idx_t = np.concatenate([np.random.choice(X.shape[2], N_t_s, replace=False)]) + idx_t = idx_t.astype(np.int32) + + X1 = X[idx_s_x, idx_s_y, :] + X2 = X1[:, idx_t] + Y1 = Y[idx_s_x, idx_s_y, :] + Y2 = Y1[:, idx_t] + T1 = T[idx_s_x, idx_s_y, :] + T2 = T1[:, idx_t] + exact_u1 = exact_u[idx_s_x, idx_s_y, :] + exact_u2 = exact_u1[:, idx_t] + exact_v1 = exact_v[idx_s_x, idx_s_y, :] + exact_v2 = exact_v1[:, idx_t] + + X_star_meas = np.hstack((X2.flatten()[:, None], Y2.flatten()[:, None], T2.flatten()[:, None])) + u_star_meas = exact_u2.flatten()[:, None] + v_star_meas = exact_v2.flatten()[:, None] + + # Prepare the training datasets + N_u_train = int(N_uv_s*N_t_s) + idx_train = np.random.choice(X_star_meas.shape[0], N_u_train, replace=False) + X_star_train = X_star_meas[idx_train, :] + u_star_train = u_star_meas[idx_train, :] + v_star_train = v_star_meas[idx_train, :] + + # Collocation points + N_f = 20000 + X_f = lb + (ub-lb)*lhs(3, N_f) + X_f = np.vstack((X_f, X_star_train)) + + + # Add Noise + noise = 0.0 + u_star_train = u_star_train + noise*np.std(u_star_train)*np.random.randn(u_star_train.shape[0], + u_star_train.shape[1]) + v_star_train = v_star_train + noise*np.std(v_star_train)*np.random.randn(v_star_train.shape[0], + v_star_train.shape[1]) + + return X_star_train, u_star_train, v_star_train, X_f + +def print_pde(lambda_uux, lambda_vuy, w, rhs_description, ut='u_t'): + pde = ut + ' = ' + "(%05f)" % (lambda_uux.real) + rhs_description[0] + "\n" + "(%05f)" % (lambda_vuy.real) + rhs_description[1] + "\n" + first = True + for i in range(len(w)): + if w[i] != 0: + if not first: + pde = pde + ' + ' + pde = pde + "(%05f)" % (w[i].real) + rhs_description[i+2] + "\n " + first = False + print(pde) diff --git a/MindFlow/applications/research/ICNet/src/derivatives.py b/MindFlow/applications/research/ICNet/src/derivatives.py new file mode 100644 index 0000000000000000000000000000000000000000..92d60ef316420a2b20d4c3c224137fb00da4c620 --- /dev/null +++ b/MindFlow/applications/research/ICNet/src/derivatives.py @@ -0,0 +1,499 @@ +# Copyright 2021 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================ +# pylint: disable=C0103 +""" +derivative +""" +from inspect import isfunction +import numpy as np +import mindspore +import mindspore.numpy as mnp +from mindspore import Tensor, nn, ops, context +from mindspore.ops import constexpr +from mindspore import dtype as mstype + + +def check_mode(api_name): + if context.get_context("mode") == context.PYNATIVE_MODE: + raise RuntimeError("{} is only supported GRAPH_MODE now but got PYNATIVE_MODE".format(api_name)) + + +def check_type(param, param_name, param_type, exclude=None): + if (exclude and isinstance(param, exclude)) or not isinstance(param, param_type): + raise TypeError("The type of {} should be instance of {}, but got {}".format( + param_name, param_type, type(param))) + +def _transfer_tensor_to_tuple(inputs): + """ + If the input is a tensor, convert it to a tuple. If not, the output is unchanged. + """ + if isinstance(inputs, Tensor): + return (inputs,) + return inputs + + +class _GenerateMultiSens(nn.Cell): + """generate sens for multi-outputs""" + def construct(self, o, net_out, sens): + if len(net_out) == 1: + return sens + all_sens = () + for i in range(len(net_out)): + if i != o: + all_sens += (mnp.zeros(net_out[i].shape, mnp.float32),) + else: + all_sens += (sens,) + return all_sens + + +class _MergeOutput(nn.Cell): + """merge output""" + def construct(self, out_tmp, gout, iters): + for i in range(iters): + out_tmp[i] = out_tmp[i] + (gout[i],) + return out_tmp + + +@constexpr +def _generate_sens(batch_size, out_chanel, i): + sens = np.zeros((batch_size, out_chanel), np.float32) + sens[:, i] = 1 + return Tensor(sens) + + +@constexpr +def _generate_indices(j): + return Tensor([j], mindspore.int32) + + +@constexpr +def _check_type(net_in, net_out, input_idx=None, output_idx=None): + """check type of input""" + if net_in is not None: + raise TypeError("The Type of network input should be Tensor but got {}".format(type(net_in))) + if input_idx is not None and (not isinstance(input_idx, int) or isinstance(input_idx, bool)): + raise TypeError("The Type of column index of input should be int but got {}".format(type(input_idx))) + if output_idx is not None and (not isinstance(output_idx, int) or isinstance(output_idx, bool)): + raise TypeError("The Type of column index of output should be int but got {}".format(type(output_idx))) + if net_out is not None: + raise TypeError("The Type of network output should be Tensor but got {}".format(type(net_out))) + + +@constexpr +def _check_dimension(in_shape, out_shape, in_idx, out_idx): + """check dimension of input""" + if len(in_shape) != 2: + raise ValueError("The dimension of network input should be 2, but got {}".format(len(in_shape))) + if len(out_shape) != 2: + raise ValueError("The dimension of network output should be 2, but got {}".format(len(out_shape))) + if in_idx is not None and out_idx is not None: + if in_idx >= in_shape[1]: + raise ValueError("input index should be in range (0, {}), but got {}".format(in_shape[1], in_idx)) + if out_idx >= out_shape[1]: + raise ValueError("output index should be in range (0, {}), but got {}".format(out_shape[1], out_idx)) + + +class Grad(nn.Cell): + """ + Computes and returns the gradients of the specified column of outputs with respect to the specified column of + inputs. + + Args: + model (Cell): a function or network that takes Tensor inputs. + argnum (int): specifies which input the output takes the first derivative of. Default: ``0``. + + Inputs: + - **x** (list) - The input is variable-length argument. The first input is a 2D network inputs (Tensor), + the last three inputs are column index of input (int), column index of output (int) and output of + network (Tensor). + + Outputs: + Tensor. The gradients of the specified column of outputs with respect to the specified column of + inputs. + + Raises: + TypeError: If the type of `argnum` is not int. + + Supported Platforms: + ``Ascend`` + + Examples: + >>> import numpy as np + >>> from mindspore import nn, Tensor + ... + >>> class Net(nn.Cell): + ... def __init__(self): + ... super(Net, self).__init__() + ... def construct(self, x): + ... return x * x + ... + >>> x = Tensor(np.array([[1.0, -2.0], [-3.0, 4.0]]).astype(np.float32)) + >>> net = Net() + >>> out = net(x) + >>> grad = Grad(net) + >>> print(grad(x, 0, 0, out).asnumpy()) + [[ 2.] + [-6.]] + """ + def __init__(self, model, argnum=0): + super(Grad, self).__init__() + check_mode("Grad") + if not isinstance(model, nn.Cell) and not isfunction(model): + raise TypeError("The type of model should be a function or network, but got {}".format(type(model))) + self.model = model + if isinstance(argnum, bool) or not isinstance(argnum, int): + raise TypeError("The type of argnum should be int, but get {}".format(type(argnum))) + self.argnum = argnum + self.grad = ops.GradOperation(get_all=True, sens_param=True) + self.gather = ops.Gather() + self.cast = ops.Cast() + self.dtype = ops.DType() + + def construct(self, *x): + """Compute the gradient of the function.""" + x = _transfer_tensor_to_tuple(x) + input_idx, output_idx, net_out = x[-3], x[-2], x[-1] + net_in = x[:-3] + _check_type(net_in[0], net_out, input_idx, output_idx) + if net_out is None: + net_out = self.model(*net_in) + net_out = _transfer_tensor_to_tuple(net_out)[0] + _check_dimension(net_in[self.argnum].shape, net_out.shape, input_idx, output_idx) + batch_size, out_chanel = net_out.shape + sens = _generate_sens(batch_size, out_chanel, output_idx) + gradient_function = self.grad(self.model) + sens = self.cast(sens, self.dtype(net_out)) + gradient = gradient_function(*net_in, sens) + if input_idx is None: + output = gradient[self.argnum] + else: + out_indices = _generate_indices(input_idx) + output = self.gather(gradient[self.argnum], out_indices, 1) + return output + + +class SecondOrderGrad(nn.Cell): + """ + Computes and returns the second order gradients of the specified column of outputs with respect to the specified + column of inputs. + + Args: + model (Cell): a function or network that takes a single Tensor input and returns a single Tensor. + input_idx1 (int): specifies the column index of input to take the first derivative, + takes values in [0, model input size - 1]. + input_idx2 (int): specifies the column index of input to take the second derivative, + takes values in [0, model input size - 1]. + output_idx (int): specifies the column index of output, takes values in [0, model output size - 1]. + + Inputs: + - **input** - The input of given function or network `model`. + + Outputs: + Tensor. + + Raises: + TypeError: If the type of `input_idx1`, `input_idx2` or `output_idx` is not int. + + Supported Platforms: + ``Ascend`` + + Examples: + >>> import numpy as np + >>> from mindspore import nn, Tensor + >>> class Net(nn.Cell): + ... def __init__(self): + ... super(Net, self).__init__() + ... + ... def construct(self, x): + ... return x * x * x + >>> x = Tensor(np.array([[1.0, -2.0], [-3.0, 4.0]]).astype(np.float32)) + >>> net = Net() + >>> out = net(x) + >>> grad = SecondOrderGrad(net, 0, 0, 0) + >>> print(grad(x).asnumpy()) + [[ 6.] + [-18.]] + """ + def __init__(self, model, input_idx1, input_idx2, output_idx): + super(SecondOrderGrad, self).__init__() + check_mode("SecondOrderGrad") + if not isinstance(model, nn.Cell) and not isfunction(model): + raise TypeError("The type of model should be a function or network, but got {}".format(type(model))) + if isinstance(input_idx1, bool) or not isinstance(input_idx1, int): + raise TypeError("The type of input_idx1 should be int, but got {}".format(type(input_idx1))) + if isinstance(input_idx2, bool) or not isinstance(input_idx2, int): + raise TypeError("The type of input_idx1 should be int, but got {}".format(type(input_idx2))) + if isinstance(output_idx, bool) or not isinstance(output_idx, int): + raise TypeError("The type of input_idx1 should be int, but got {}".format(type(output_idx))) + self.jac1 = _FirstOrderGrad(model, input_idx=input_idx1, output_idx=output_idx) + self.jac2 = _FirstOrderGrad(self.jac1, input_idx=input_idx2, output_idx=0) + + def construct(self, x): + hes = self.jac2(x) + return hes + +class ThirdOrderGrad_1d(nn.Cell): + """ + Computes and returns the Third order gradients of the specified column of outputs with respect to the specified + column of inputs. + """ + def __init__(self, model, input_idx1, input_idx2, output_idx): + super(ThirdOrderGrad_1d, self).__init__() + check_mode("SecondOrderGrad") + if not isinstance(model, nn.Cell) and not isfunction(model): + raise TypeError("The type of model should be a function or network, but got {}".format(type(model))) + if isinstance(input_idx1, bool) or not isinstance(input_idx1, int): + raise TypeError("The type of input_idx1 should be int, but got {}".format(type(input_idx1))) + if isinstance(input_idx2, bool) or not isinstance(input_idx2, int): + raise TypeError("The type of input_idx1 should be int, but got {}".format(type(input_idx2))) + if isinstance(output_idx, bool) or not isinstance(output_idx, int): + raise TypeError("The type of input_idx1 should be int, but got {}".format(type(output_idx))) + self.jac1 = _FirstOrderGrad(model, input_idx=input_idx1, output_idx=output_idx) + self.jac2 = _FirstOrderGrad(self.jac1, input_idx=input_idx2, output_idx=0) + self.jac3 = _FirstOrderGrad(self.jac2, input_idx=input_idx2, output_idx=0) + + def construct(self, x): + hes = self.jac3(x) + return hes + +class ForthOrderGrad_1d(nn.Cell): + """ + Computes and returns the Third order gradients of the specified column of outputs with respect to the specified + column of inputs. + """ + def __init__(self, model, input_idx1, input_idx2, output_idx): + super(ForthOrderGrad_1d, self).__init__() + check_mode("SecondOrderGrad") + if not isinstance(model, nn.Cell) and not isfunction(model): + raise TypeError("The type of model should be a function or network, but got {}".format(type(model))) + if isinstance(input_idx1, bool) or not isinstance(input_idx1, int): + raise TypeError("The type of input_idx1 should be int, but got {}".format(type(input_idx1))) + if isinstance(input_idx2, bool) or not isinstance(input_idx2, int): + raise TypeError("The type of input_idx1 should be int, but got {}".format(type(input_idx2))) + if isinstance(output_idx, bool) or not isinstance(output_idx, int): + raise TypeError("The type of input_idx1 should be int, but got {}".format(type(output_idx))) + self.jac1 = _FirstOrderGrad(model, input_idx=input_idx1, output_idx=output_idx) + self.jac2 = _FirstOrderGrad(self.jac1, input_idx=input_idx2, output_idx=0) + self.jac3 = _FirstOrderGrad(self.jac2, input_idx=input_idx2, output_idx=0) + self.jac4 = _FirstOrderGrad(self.jac3, input_idx=input_idx2, output_idx=0) + + def construct(self, x): + hes = self.jac4(x) + return hes + + +class Jacobian(nn.Cell): + r""" + Computes the Jacobian of a given function or network. + + Note: + The output of the given function or network should be a single Tensor. + + Args: + net (Union[function, Cell]): a function or network that takes Tensor inputs. + arg_nums (int): specifies which input the output takes the first derivative of. + out_idx (int): specifies which output to take the first derivative of. + + Inputs: + - **x** (Tensor) - The inputs of the function or network `net`. + + Outputs: + Tensor or tuple of Tensors. If `arg_nums` is int, output will be a Tensor whose shape is the shape of + specified output * the shape of specified input. If `arg_nums` is None, output will be a tuple of Tensors + where output[i] will contain the Jacobian of the specified output and ith input and will have as size the + concatenation of the sizes of the corresponding output and the corresponding input. + + Raises: + TypeError: if the type of `arg_nums` or `out_idx` is not int. + + Supported Platforms: + ``Ascend`` + + Examples: + >>> import numpy as np + >>> from mindspore import Tensor + >>> def func(x, y): + >>> return (x * x * x * y + 3 * y * y * x).sum() + ... + >>> a = Tensor(np.array([[1, 3], [5, 9], [8, 2]], np.float32)) + >>> b = Tensor(np.array([[4, 6], [7, 2], [2, 1]], np.float32)) + >>> jac = Jacobian(func, 0, 0) + >>> output = jac(a, b) + >>> print(output.shape) + (3, 2) + """ + def __init__(self, net, argnums=0, out_idx=0): + super(Jacobian, self).__init__() + if not (isinstance(argnums, int) or argnums is None) or not isinstance(out_idx, int): + raise TypeError("The type of argnums should be int or None and out_idx should be int.") + self.net = net + self.argnums = argnums + self.out_idx = out_idx + self.grad_op = ops.GradOperation(get_all=True, sens_param=True) + self.eye = ops.Eye() + self.concat = ops.Concat() + self.reshape = ops.Reshape() + self.tuple_len = ops.tuple_len + self._merge_output = _MergeOutput() + self._generate_multi_sens = _GenerateMultiSens() + + def construct(self, *x): + """ + forward + + Args: + inputs (tuple): input tensor. + """ + net_out = _transfer_tensor_to_tuple(self.net(*x)) + net_out_target = net_out[self.out_idx] + grad_fn = self.grad_op(self.net) + input_len = self.tuple_len(x) + + identity_matrix = self.eye(net_out_target.size, net_out_target.size, mstype.float32) + identity_matrix = ops.Split(0, net_out_target.size)(identity_matrix) + if self.argnums is None: + out_tmp = [()] * input_len + for line in identity_matrix: + sens = self.reshape(line, net_out_target.shape) + grad_wrt_output = self._generate_multi_sens(self.out_idx, net_out, sens) + grad = grad_fn(*x, grad_wrt_output) + out_tmp = self._merge_output(out_tmp, grad, input_len) + output = () + for i in range(input_len): + out_tmp[i] = self.concat(out_tmp[i]) + output = output + (self.reshape(out_tmp[i], net_out_target.shape + x[i].shape),) + return output + output = () + for line in identity_matrix: + sens = self.reshape(line, net_out_target.shape) + grad_wrt_output = self._generate_multi_sens(self.out_idx, net_out, sens) + grad = grad_fn(*x, grad_wrt_output) + output = output + (grad[self.argnums],) + output = self.concat(output) + return self.reshape(output, net_out_target.shape + x[self.argnums].shape) + + +class Hessian(nn.Cell): + r""" + Computes the Hessian of a given function or network. + + Note: + The output of the given function or network should be a single Tensor. + + Args: + net (Union[function, Cell]): a function or network that takes Tensor inputs and returns a single Tensor. + diff1_idx (int): specifies which input the output takes the first derivative of. + diff2_idx (int): specifies which input the output takes the second derivative of. + + Inputs: + - **x** (Tensor) - The inputs of the function or network `net`. + + Outputs: + Tensor, the shape is the shape output * shape of specified input * the shape of specified input. + + Raises: + TypeError: if the type of `diff1_idx` or `diff2_idx` is not int. + + Supported Platforms: + ``Ascend`` + + Examples: + >>> import numpy as np + >>> from mindspore import Tensor + >>> def func(x, y): + >>> return (x * x * x * y + 3 * y * y * x).sum() + >>> a = Tensor(np.array([[1, 3], [5, 9], [8, 2]], np.float32)) + >>> b = Tensor(np.array([[4, 6], [7, 2], [2, 1]], np.float32)) + >>> hes = Hessian(func, 0, 0) + >>> output = hes(a, b) + >>> print(output.shape) + (3, 2, 3, 2) + """ + def __init__(self, net, diff1_idx, diff2_idx, out_idx=0): + super(Hessian, self).__init__() + if not isinstance(diff1_idx, int) or not (isinstance(diff2_idx, int) or diff2_idx is None): + raise TypeError("The type of diff1 should be int and diff2 should be int or None.") + self.jac1 = Jacobian(net, argnums=None, out_idx=out_idx) + self.jac2 = Jacobian(self.jac1, argnums=diff2_idx, out_idx=diff1_idx) + + def construct(self, *x): + return self.jac2(*x) + + +def jacobian(func, inputs): + r""" + Function that computes the Jacobian of a given function or network. + + Parameters: + func: a function or network that takes Tensor inputs. + inputs: The inputs of the function or network `net`. + """ + inputs = _transfer_tensor_to_tuple(inputs) + func_out = _transfer_tensor_to_tuple(func(*inputs)) + output = () + for i in range(len(func_out)): + jac = Jacobian(func, argnums=None, out_idx=i) + output = output + _transfer_tensor_to_tuple(jac(*inputs)) + return output + + +def hessian(func, inputs): + r""" + Function that computes the Hessian of a given function or network. + + Parameters: + func: a function or network that takes Tensor inputs. + inputs: The inputs of the function or network `net`. + """ + inputs = _transfer_tensor_to_tuple(inputs) + func_out = _transfer_tensor_to_tuple(func(*inputs)) + output = () + for i in range(len(func_out)): + out_tmp = () + for j in range(len(inputs)): + hes = Hessian(func, diff1_idx=j, diff2_idx=None, out_idx=i) + out_tmp = out_tmp + _transfer_tensor_to_tuple(hes(*inputs)) + output = output + out_tmp + return output + + +class _FirstOrderGrad(nn.Cell): + """compute first-order derivative""" + def __init__(self, model, argnums=0, input_idx=None, output_idx=1): + super(_FirstOrderGrad, self).__init__() + self.model = model + self.argnums = argnums + self.input_idx = input_idx + self.output_idx = output_idx + self.grad = ops.GradOperation(get_all=True, sens_param=True) + self.gather = ops.Gather() + self.cast = ops.Cast() + self.dtype = ops.DType() + + def construct(self, *x): + """Defines the computation to be performed""" + x = _transfer_tensor_to_tuple(x) + _check_type(x[self.argnums], None) + net_out = self.model(*x) + net_out = _transfer_tensor_to_tuple(net_out)[0] + _check_dimension(x[self.argnums].shape, net_out.shape, self.input_idx, self.output_idx) + batch_size, out_chanel = net_out.shape + sens = _generate_sens(batch_size, out_chanel, self.output_idx) + gradient_function = self.grad(self.model) + sens = self.cast(sens, self.dtype(net_out)) + gradient = gradient_function(*x, sens) + outout_indices = _generate_indices(self.input_idx) + output = self.gather(gradient[self.argnums], outout_indices, 1) + return output diff --git a/MindFlow/applications/research/ICNet/src/network.py b/MindFlow/applications/research/ICNet/src/network.py new file mode 100644 index 0000000000000000000000000000000000000000..3b0e49999a9787f9f57e69257b270a1c2c9729e9 --- /dev/null +++ b/MindFlow/applications/research/ICNet/src/network.py @@ -0,0 +1,385 @@ +# Copyright 2023 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================ +# pylint: disable=C0103 +"""Network definitions""" +import numpy as np + +import mindspore as ms +from mindspore import nn, Tensor, ops, Parameter +from mindspore.common.api import jit +from mindspore.amp import DynamicLossScaler +from mindspore.ops.functional import norm +from src.derivatives import ForthOrderGrad_1d, ThirdOrderGrad_1d, SecondOrderGrad, Grad + + +# the deep neural network +class NeuralNet(nn.Cell): + """NeuralNetwork""" + def __init__(self, layers, msfloat_type): + super(NeuralNet, self).__init__() + + # parameters + self.depth = len(layers) - 1 + + # set up activation + self.activation = nn.Tanh() + + layer_list = [] + for i in range(self.depth - 1): + layer_list.append(nn.Dense(layers[i], layers[i+1]).to_float(msfloat_type)) + layer_list.append(self.activation.to_float(msfloat_type)) + + layer_list.append(nn.Dense(layers[-2], layers[-1]).to_float(msfloat_type)) + + # deploy layers + self.layers = nn.SequentialCell(layer_list).to_float(msfloat_type) + + @jit + def construct(self, x): + out = self.layers(x) + return out + +class InvarianceConstrainedNN: + """InvarianceConstrainedNN""" + # Initialize the class + def __init__(self, X, u, X_f, layers, batchno, use_npu, msfloat_type): + + # The Size of Network + self.layers = layers + + self.use_npu = use_npu + + self.msfloat_type = msfloat_type + + # Initialize trainable parameters + self.lambda_u = Tensor(np.zeros([5, 1]), self.msfloat_type) + self.lambda_uux = Tensor([0.0], self.msfloat_type) + + self.lambda_u = Parameter(self.lambda_u, name="name_u", requires_grad=True) + self.lambda_uux = Parameter(self.lambda_uux, name="name_uux", requires_grad=True) + + # Training data + self.x = Tensor(np.array(X[:, 0:1], np.float32), self.msfloat_type) + self.t = Tensor(np.array(X[:, 1:2], np.float32), self.msfloat_type) + self.u = Tensor(np.array(u, np.float32), self.msfloat_type) + + # Collection data for physics + self.x_f = Tensor(np.array(X_f[:, 0:1], np.float32), self.msfloat_type) + self.t_f = Tensor(np.array(X_f[:, 1:2], np.float32), self.msfloat_type) + + self.batchno = batchno + self.batchsize_f = np.floor(self.x_f.shape[0]/self.batchno) + + # Initialize NNs---deep neural networks + self.dnn = NeuralNet(layers, self.msfloat_type) + + params = self.dnn.trainable_params() + params.append(self.lambda_u) + params.append(self.lambda_uux) + + # The function of Auto-differentiation + self.grad = Grad(self.dnn) + self.hessian_u_xx = SecondOrderGrad(self.dnn, 0, 0, output_idx=0) + self.hessian_u_xxx = ThirdOrderGrad_1d(self.dnn, 0, 0, output_idx=0) + self.hessian_u_xxxx = ForthOrderGrad_1d(self.dnn, 0, 0, output_idx=0) + + # The loss function + self.l2_loss = nn.MSELoss() + self.reduce_mean = ops.ReduceMean() + + # optimizers: using the same settings + self.loss_scaler = DynamicLossScaler(1024, 2, 100) + + def net_u(self, x, t): + H = ops.concat([x, t], 1) + u = self.dnn(H) + return u + + def net_f(self, x, t, N_f): + """ The minspore autograd version of calculating residual """ + u = self.net_u(x, t) + data = ops.concat([x, t], 1) + + u_t = self.grad(data, 1, 0, u) + u_x = self.grad(data, 0, 0, u) + u_xx = self.hessian_u_xx(data) + u_xxx = self.hessian_u_xxx(data) + u_xxxx = self.hessian_u_xxxx(data) + + derivatives = [Tensor(np.ones((int(N_f), 1), dtype=np.float32), dtype=self.msfloat_type)] + derivatives.append(u_x) + derivatives.append(u_xx) + derivatives.append(u_xxx) + derivatives.append(u_xxxx) + + f_u = u_t + self.lambda_uux * u * u_x + + lib_fun = derivatives + for i in range(len(lib_fun)): + f_u = f_u - lib_fun[i]*self.lambda_u[i:i+1, 0:1] + + return f_u + + def loss_fn(self, x, t, x_f, t_f, u): + """loss_fn""" + u_pred = self.net_u(x, t) + f_u_pred = self.net_f(x_f, t_f, self.batchsize_f) + + loss_u = self.l2_loss(u_pred, u) + loss_f_u = self.l2_loss(f_u_pred, ms.ops.zeros_like(f_u_pred)) + lambda_u = self.lambda_u + loss_lambda_u = 1e-7*norm(lambda_u, ord=1) + + loss = loss_u + loss_f_u + loss_lambda_u + + if self.use_npu: + loss = self.loss_scaler.scale(loss) + + return loss, loss_u, loss_f_u, loss_lambda_u + +class InvarianceConstrainedNN_STRdige: + """InvarianceConstrainedNN_STRdige""" + # Initialize the class + def __init__(self, X, u, X_f, layers, batchno, lambda_u, lambda_uux, load_params, second_path, msfloat_type): + + # The Size of Network + self.layers = layers + + self.msfloat_type = msfloat_type + + # Initialize trainable parameters + self.lambda_u = Tensor(lambda_u, self.msfloat_type) + self.lambda_uux = Tensor(lambda_uux, self.msfloat_type) + + self.lambda_u = Parameter(self.lambda_u, name="name_u", requires_grad=True) + self.lambda_uux = Parameter(self.lambda_uux, name="name_uux", requires_grad=True) + + # Training data + self.x = Tensor(np.array(X[:, 0:1], np.float32), self.msfloat_type) + self.t = Tensor(np.array(X[:, 1:2], np.float32), self.msfloat_type) + self.u = Tensor(np.array(u, np.float32), self.msfloat_type) + + # Collection data for physics + self.x_f = Tensor(np.array(X_f[:, 0:1], np.float32), self.msfloat_type) + self.t_f = Tensor(np.array(X_f[:, 1:2], np.float32), self.msfloat_type) + + self.batchno = batchno + self.batchsize_f = np.floor(self.x_f.shape[0]/self.batchno) + + # Initialize NNs---deep neural networks + self.dnn = NeuralNet(layers, self.msfloat_type) + if load_params: + params_dict = ms.load_checkpoint(f'model/{second_path}/model.ckpt') + ms.load_param_into_net(self.dnn, params_dict) + + params = self.dnn.trainable_params() + params.append(self.lambda_u) + params.append(self.lambda_uux) + + # The function of Auto-differentiation + self.grad = Grad(self.dnn) + self.hessian_u_xx = SecondOrderGrad(self.dnn, 0, 0, output_idx=0) + self.hessian_u_xxx = ThirdOrderGrad_1d(self.dnn, 0, 0, output_idx=0) + self.hessian_u_xxxx = ForthOrderGrad_1d(self.dnn, 0, 0, output_idx=0) + + # The loss function + self.l2_loss = nn.MSELoss() + self.reduce_mean = ops.ReduceMean() + + # optimizers: using the same settings + self.optimizer_Adam = nn.Adam(params, learning_rate=1e-3) + self.loss_scaler = DynamicLossScaler(1024, 2, 100) + + def net_u(self, x, t): + H = ops.concat([x, t], 1) + u = self.dnn(H) + return u + + def net_f(self, x, t, N_f): + """ The minspore autograd version of calculating residual """ + u = self.net_u(x, t) + data = ops.concat([x, t], 1) + + u_t = self.grad(data, 1, 0, u) + u_x = self.grad(data, 0, 0, u) + u_xx = self.hessian_u_xx(data) + u_xxx = self.hessian_u_xxx(data) + u_xxxx = self.hessian_u_xxxx(data) + + derivatives = [Tensor(np.ones((int(N_f), 1), dtype=np.float32), dtype=self.msfloat_type)] + derivatives.append(u_x) + derivatives.append(u_xx) + derivatives.append(u_xxx) + derivatives.append(u_xxxx) + + f_u = u_t + self.lambda_uux * u * u_x + Phi = ms.ops.cat((Tensor(np.ones((int(N_f), 1), dtype=np.float32), dtype=self.msfloat_type), + u_x, u_xx, u_xxx, u_xxxx), 1) + + lib_fun = derivatives + for i in range(len(lib_fun)): + f_u = f_u - lib_fun[i]*self.lambda_u[i:i+1, 0:1] + + return f_u, Phi, u_t, u*u_x + + def loss_fn(self, x, t, x_f, t_f, u): + """loss_fn""" + u_pred = self.net_u(x, t) + f_u_pred, _, _ = self.net_f(x_f, t_f, self.batchsize_f) + + loss_u = self.l2_loss(u_pred, u) + loss_f_u = self.l2_loss(f_u_pred, ops.zeros_like(f_u_pred)) + lambda_u = self.lambda_u + loss_lambda_u = 1e-7*norm(lambda_u, ord=1) + + loss = loss_u + loss_f_u + loss_lambda_u + return loss, loss_u, loss_f_u, loss_lambda_u + + def call_trainstridge(self, lam, d_tol): + _, Phi_u, u_t, uu_x = self.net_f(self.x_f, self.t_f, self.batchsize_f) + + U_t_pred = u_t + self.lambda_uux * uu_x + + lambda_u2 = self.train_stridge(Phi_u.numpy(), U_t_pred.numpy(), lam, d_tol, maxit=100, STR_iters=10, l0_penalty=None, normalize=2, split=0.8, print_best_tol=False, uv_flag=True) + + return lambda_u2 + + def train_stridge(self, R, Ut, lam, d_tol, maxit, STR_iters=10, l0_penalty=None, normalize=2,\ + split=0.8, print_best_tol=False, uv_flag=True): + """ + Inspired by Rudy, Samuel H., et al. "Data-driven discovery of partial differential equations." + Science Advances 3.4 (2017): e1602614. + """ + + # Split data into 80% training and 20% test, then search for the best tolderance. + np.random.seed(0) # for consistency + n, _ = R.shape + train = np.random.choice(n, int(n*split), replace=False) + test = [i for i in np.arange(n) if i not in train] + TestR = R[test, :] + TestY = Ut[test, :] + + # Set up the initial tolerance and l0 penalty + d_tol = float(d_tol) + tol = d_tol + + # Get the standard least squares estimator + if uv_flag: + w_best = self.lambda_u.numpy() + else: + w_best = self.lambda_u.numpy() + + err_f = np.mean((TestY - TestR.dot(w_best))**2) + + if uv_flag: + self.l0_penalty_0_u = 10*err_f + l0_penalty = self.l0_penalty_0_u + else: + self.l0_penalty_0_v = 10*err_f + l0_penalty = self.l0_penalty_0_v + + err_lambda = l0_penalty*np.count_nonzero(w_best) + err_best = err_f + err_lambda + tol_best = 0 + + # Now increase tolerance until test performance decreases + for iteration in range(maxit): + print(f"Iteration {iteration + 1}") + + # Get a set of coefficients and error + w = self.stridge(R, Ut, lam, STR_iters, tol, normalize=normalize, uv_flag=uv_flag) + err_f = np.mean((TestY - TestR.dot(w))**2) + + err_lambda = l0_penalty*np.count_nonzero(w) + err = err_f + err_lambda + + # Has the accuracy improved? + if err <= err_best: + err_best = err + w_best = w + tol_best = tol + tol = 1.2*tol + + else: + tol = 0.8*tol + + if print_best_tol: print("Optimal tolerance:", tol_best) + + optimaltol_history = np.empty([0]) + optimaltol_history = np.append(optimaltol_history, tol_best) + + return np.real(w_best) + + def stridge(self, X0, y, lam, maxit, tol, normalize=2, uv_flag=True): + """stridge""" + n, d = X0.shape + X = np.zeros((n, d), dtype=np.float32) + # First normalize data + if normalize != 0: + Mreg = np.zeros((d, 1)) + for i in range(0, d): + Mreg[i] = 1.0/(np.linalg.norm(X0[:, i], normalize)) + X[:, i] = Mreg[i]*X0[:, i] + else: X = X0 + + # Get the standard ridge estimate, inherit w from previous training + if uv_flag: + w = self.lambda_u.numpy()/Mreg + else: + w = self.lambda_u.numpy()/Mreg + + num_relevant = d + biginds = np.where(abs(w) > tol)[0] + + # Threshold and continue + for j in range(maxit): + + # Figure out which items to cut out + smallinds = np.where(abs(w) < tol)[0] + new_biginds = [i for i in range(d) if i not in smallinds] + + # If nothing changes then stop + if num_relevant == len(new_biginds): + break + else: num_relevant = len(new_biginds) + + # Also make sure we didn't just lose all the coefficients + if not new_biginds: + if j == 0: + if normalize != 0: + w = np.multiply(Mreg, w) + return w + return w + break + biginds = new_biginds + + # Otherwise get a new guess + w[smallinds] = 0 + + if lam != 0: + w[biginds] = np.linalg.lstsq(X[:, biginds].T.dot(X[:, biginds]) + lam*np.eye(len(biginds)), + X[:, biginds].T.dot(y))[0] + else: + w[biginds] = np.linalg.lstsq(X[:, biginds], y)[0] + + # Now that we have the sparsity pattern, use standard least squares to get w + if biginds != []: + w[biginds] = np.linalg.lstsq(X[:, biginds].T.dot(X[:, biginds]) + lam*np.eye(len(biginds)), + X[:, biginds].T.dot(y))[0] + + if normalize != 0: + w = np.multiply(Mreg, w) + return w + return w diff --git a/MindFlow/applications/research/ICNet/src/network_CKG.py b/MindFlow/applications/research/ICNet/src/network_CKG.py new file mode 100644 index 0000000000000000000000000000000000000000..23fc371d1592cb4ede837cf46fdbbdc95e556993 --- /dev/null +++ b/MindFlow/applications/research/ICNet/src/network_CKG.py @@ -0,0 +1,493 @@ +# Copyright 2023 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================ +# pylint: disable=C0103 +"""Network definitions""" +import numpy as np + +import mindspore as ms +from mindspore import nn, Tensor, ops, Parameter +from mindspore.common.api import jit +from mindspore.amp import DynamicLossScaler +from mindspore.ops.functional import norm +from src.derivatives import SecondOrderGrad, Grad + + +# the deep neural network +class NeuralNet(nn.Cell): + """NeuralNetwork""" + def __init__(self, layers, msfloat_type): + super(NeuralNet, self).__init__() + + # parameters + self.depth = len(layers) - 1 + + # set up activation + self.activation = nn.Tanh() + + layer_list = [] + for i in range(self.depth - 1): + layer_list.append(nn.Dense(layers[i], layers[i + 1]).to_float(msfloat_type)) + layer_list.append(self.activation.to_float(msfloat_type)) + + layer_list.append(nn.Dense(layers[-2], layers[-1]).to_float(msfloat_type)) + + # deploy layers + self.layers = nn.SequentialCell(layer_list).to_float(msfloat_type) + + @jit + def construct(self, x): + out = self.layers(x) + return out + + +class InvarianceConstrainedNN: + """InvarianceConstrainedNN""" + # Initialize the class + def __init__(self, X, u, v, X_f, layers, batchno, use_npu, msfloat_type): + + # The Size of Network + self.layers = layers + + self.use_npu = use_npu + + self.msfloat_type = msfloat_type + + # Initialize trainable parameters + self.lambda_u = Tensor(np.zeros([10, 1]), self.msfloat_type) + self.lambda_v = Tensor(np.zeros([10, 1]), self.msfloat_type) + + self.lambda_uxx = Tensor([0.0], self.msfloat_type) + self.lambda_uyy = Tensor([0.0], self.msfloat_type) + self.lambda_vxx = Tensor([0.0], self.msfloat_type) + self.lambda_vyy = Tensor([0.0], self.msfloat_type) + + self.lambda_u = Parameter(self.lambda_u, name="name_u", requires_grad=True) + self.lambda_v = Parameter(self.lambda_v, name="name_v", requires_grad=True) + + self.lambda_uxx = Parameter(self.lambda_uxx, name="name_uxx", requires_grad=True) + self.lambda_uyy = Parameter(self.lambda_uyy, name="name_uyy", requires_grad=True) + self.lambda_vxx = Parameter(self.lambda_vxx, name="name_vxx", requires_grad=True) + self.lambda_vyy = Parameter(self.lambda_vyy, name="name_vyy", requires_grad=True) + + # Training data + self.x = Tensor(np.array(X[:, 0:1], np.float32), self.msfloat_type) + self.y = Tensor(np.array(X[:, 1:2], np.float32), self.msfloat_type) + self.t = Tensor(np.array(X[:, 2:3], np.float32), self.msfloat_type) + self.u = Tensor(np.array(u, np.float32), self.msfloat_type) + self.v = Tensor(np.array(v, np.float32), self.msfloat_type) + + # Collection data for physics + self.x_f = Tensor(np.array(X_f[:, 0:1], np.float32), self.msfloat_type) + self.y_f = Tensor(np.array(X_f[:, 1:2], np.float32), self.msfloat_type) + self.t_f = Tensor(np.array(X_f[:, 2:3], np.float32), self.msfloat_type) + + self.batchno = batchno + self.batchsize_f = np.floor(self.x_f.shape[0] / self.batchno) + + # Initialize NNs---deep neural networks + self.dnn = NeuralNet(layers, self.msfloat_type) + + params = self.dnn.trainable_params() + params.append(self.lambda_u) + params.append(self.lambda_v) + + params.append(self.lambda_uxx) + params.append(self.lambda_uyy) + params.append(self.lambda_vxx) + params.append(self.lambda_vyy) + + # The function of Auto-differentiation + self.grad = Grad(self.dnn) + self.hessian_u_tt = SecondOrderGrad(self.dnn, 2, 2, output_idx=0) + self.hessian_u_xx = SecondOrderGrad(self.dnn, 0, 0, output_idx=0) + self.hessian_u_yy = SecondOrderGrad(self.dnn, 1, 1, output_idx=0) + + self.hessian_v_tt = SecondOrderGrad(self.dnn, 2, 2, output_idx=1) + self.hessian_v_xx = SecondOrderGrad(self.dnn, 0, 0, output_idx=1) + self.hessian_v_yy = SecondOrderGrad(self.dnn, 1, 1, output_idx=1) + + # The loss function + self.l2_loss = nn.MSELoss() + self.reduce_mean = ops.ReduceMean() + + # optimizers: using the same settings + self.loss_scaler = DynamicLossScaler(1024, 2, 100) + + def net_u(self, x, y, t): + H = ops.concat([x, y, t], 1) + Y = self.dnn(H) + return Y + + def net_f(self, x_f, y_f, t_f, N_f): + """ The minspore autograd version of calculating residual """ + U = self.net_u(x_f, y_f, t_f) + u = U[:, 0:1] + v = U[:, 1:2] + + data = ops.concat([x_f, y_f, t_f], 1) + + u_tt = self.hessian_u_tt(data) + u_xx = self.hessian_u_xx(data) + u_yy = self.hessian_u_yy(data) + + v_tt = self.hessian_v_tt(data) + v_xx = self.hessian_v_xx(data) + v_yy = self.hessian_v_yy(data) + + lib_fun = [ + Tensor( + np.ones( + (int(N_f), + 1), + dtype=np.float32), + self.msfloat_type), + u, + u * u, + u * u * u, + u * v, + u * v * v, + v, + v * v, + v * v * v, + v * u * u] + + f_u = u_tt + self.lambda_uxx * u_xx + self.lambda_uyy * u_yy + f_v = v_tt + self.lambda_vxx * v_xx + self.lambda_vyy * v_yy + + for i in range(len(lib_fun)): + f_u = f_u - lib_fun[i] * self.lambda_u[i:i + 1, 0:1] + f_v = f_v - lib_fun[i] * self.lambda_v[i:i + 1, 0:1] + + return f_u, f_v + + def loss_fn(self, x, y, t, x_f, y_f, t_f, u, v): + """loss_fn""" + U = self.net_u(x, y, t) + u_pred = U[:, 0:1] + v_pred = U[:, 1:2] + + f_u_pred, f_v_pred = self.net_f(x_f, y_f, t_f, self.batchsize_f) + + loss_u = self.l2_loss(u_pred, u) + loss_v = self.l2_loss(v_pred, v) + + loss_f_u = self.l2_loss(f_u_pred, ops.zeros_like(f_u_pred)) + loss_f_v = self.l2_loss(f_v_pred, ops.zeros_like(f_v_pred)) + + lambda_u = self.lambda_u + lambda_v = self.lambda_v + loss_lambda_u = 1e-7 * norm(lambda_u, ord=1) + loss_lambda_v = 1e-7 * norm(lambda_v, ord=1) + + loss = loss_u + loss_v + loss_f_u + loss_f_v + loss_lambda_u + loss_lambda_v + + if self.use_npu: + loss = self.loss_scaler.scale(loss) + return loss, loss_u, loss_v, loss_f_u, loss_f_v, loss_lambda_u, loss_lambda_v + + +class InvarianceConstrainedNN_STRdige: + """InvarianceConstrainedNN_STRdige""" + # Initialize the class + def __init__(self, X, u, v, X_f, layers, batchno, lambda_u, lambda_v, lambda_uxx, + lambda_uyy, lambda_vxx, lambda_vyy, load_params, second_path, msfloat_type): + + # The Size of Network + self.layers = layers + + self.msfloat_type = msfloat_type + self.npfloat_type = npfloat_type + + # Initialize trainable parameters + self.lambda_u = Tensor(lambda_u, self.msfloat_type) + self.lambda_v = Tensor(lambda_v, self.msfloat_type) + self.lambda_uxx = Tensor(lambda_uxx, self.msfloat_type) + self.lambda_uyy = Tensor(lambda_uyy, self.msfloat_type) + self.lambda_vxx = Tensor(lambda_vxx, self.msfloat_type) + self.lambda_vyy = Tensor(lambda_vyy, self.msfloat_type) + + self.lambda_u = Parameter(self.lambda_u, name="name_u", requires_grad=True) + self.lambda_v = Parameter(self.lambda_v, name="name_v", requires_grad=True) + self.lambda_uxx = Parameter(self.lambda_uxx, name="name_uxx", requires_grad=True) + self.lambda_uyy = Parameter(self.lambda_uyy, name="name_uyy", requires_grad=True) + self.lambda_vxx = Parameter(self.lambda_vxx, name="name_vxx", requires_grad=True) + self.lambda_vyy = Parameter(self.lambda_vyy, name="name_vyy", requires_grad=True) + + # Training data + self.x = Tensor(np.array(X[:, 0:1], np.float32), self.msfloat_type) + self.y = Tensor(np.array(X[:, 1:2], np.float32), self.msfloat_type) + self.t = Tensor(np.array(X[:, 2:3], np.float32), self.msfloat_type) + self.u = Tensor(np.array(u, np.float32), self.msfloat_type) + self.v = Tensor(np.array(v, np.float32), self.msfloat_type) + + # Collection data for physics + self.x_f = Tensor(np.array(X_f[:, 0:1], np.float32), self.msfloat_type) + self.y_f = Tensor(np.array(X_f[:, 1:2], np.float32), self.msfloat_type) + self.t_f = Tensor(np.array(X_f[:, 2:3], np.float32), self.msfloat_type) + + self.batchno = batchno + self.batchsize_f = np.floor(self.x_f.shape[0] / self.batchno) + + # Initialize NNs---deep neural networks + self.dnn = NeuralNet(layers, self.msfloat_type) + if load_params: + params_dict = ms.load_checkpoint(f'model/{second_path}/model.ckpt') + ms.load_param_into_net(self.dnn, params_dict) + + params = self.dnn.trainable_params() + params.append(self.lambda_u) + params.append(self.lambda_v) + + params.append(self.lambda_uxx) + params.append(self.lambda_uyy) + params.append(self.lambda_vxx) + params.append(self.lambda_vyy) + + # The function of Auto-differentiation + self.grad = Grad(self.dnn) + self.hessian_u_tt = SecondOrderGrad(self.dnn, 2, 2, output_idx=0) + self.hessian_u_xx = SecondOrderGrad(self.dnn, 0, 0, output_idx=0) + self.hessian_u_yy = SecondOrderGrad(self.dnn, 1, 1, output_idx=0) + + self.hessian_v_tt = SecondOrderGrad(self.dnn, 2, 2, output_idx=1) + self.hessian_v_xx = SecondOrderGrad(self.dnn, 0, 0, output_idx=1) + self.hessian_v_yy = SecondOrderGrad(self.dnn, 1, 1, output_idx=1) + + # The loss function + self.l2_loss = nn.MSELoss() + self.reduce_mean = ops.ReduceMean() + + # optimizers: using the same settings + self.optimizer_Adam = nn.Adam(params, learning_rate=1e-3) + self.loss_scaler = DynamicLossScaler(1024, 2, 100) + + def net_u(self, x, y, t): + H = ops.concat([x, y, t], 1) + Y = self.dnn(H) + return Y + + def net_f(self, x_f, y_f, t_f, N_f): + """ The minspore autograd version of calculating residual """ + U = self.net_u(x_f, y_f, t_f) + u = U[:, 0:1] + v = U[:, 1:2] + + data = ops.concat([x_f, y_f, t_f], 1) + + u_tt = self.hessian_u_tt(data) + u_xx = self.hessian_u_xx(data) + u_yy = self.hessian_u_yy(data) + + v_tt = self.hessian_v_tt(data) + v_xx = self.hessian_v_xx(data) + v_yy = self.hessian_v_yy(data) + + f_u = u_tt + self.lambda_uxx * u_xx + self.lambda_uyy * u_yy + f_v = v_tt + self.lambda_vxx * v_xx + self.lambda_vyy * v_yy + + Phi_u = ops.cat((Tensor(np.ones((int(N_f), 1), dtype=np.float32), self.msfloat_type), + u, u * u, u * u * u, u * v, u * v * v, v, v * v, v * v * v, v * u * u), 1) + Phi_v = ops.cat((Tensor(np.ones((int(N_f), 1), dtype=np.float32), self.msfloat_type), + u, u * u, u * u * u, u * v, u * v * v, v, v * v, v * v * v, v * u * u), 1) + + return f_u, f_v, Phi_u, Phi_v + + def loss_fn(self, x, y, t, x_f, y_f, t_f, u, v): + """loss_fn""" + U = self.net_u(x, y, t) + u_pred = U[:, 0:1] + v_pred = U[:, 1:2] + + f_u_pred, f_v_pred, _, _ = self.net_f(x_f, y_f, t_f, self.batchsize_f) + + loss_u = self.l2_loss(u_pred, u) + loss_v = self.l2_loss(v_pred, v) + + loss_f_u = self.l2_loss(f_u_pred, ops.zeros_like(f_u_pred)) + loss_f_v = self.l2_loss(f_v_pred, ops.zeros_like(f_v_pred)) + + lambda_u = self.lambda_u + lambda_v = self.lambda_v + loss_lambda_u = 1e-7 * norm(lambda_u, ord=1) + loss_lambda_v = 1e-7 * norm(lambda_v, ord=1) + + loss = loss_u + loss_v + loss_f_u + loss_f_v + loss_lambda_u + loss_lambda_v + + # loss = self.loss_scaler.scale(loss) + return loss, loss_u, loss_v, loss_f_u, loss_f_v, loss_lambda_u, loss_lambda_v + + def call_trainstridge(self, lam, d_tol): + """call_trainstridge""" + U_tt_pred, V_tt_pred, Phi_u, Phi_v = self.net_f( + self.x_f, self.y_f, self.t_f, self.batchsize_f) + + lambda_u2 = self.train_stridge( + Phi_u.numpy(), + U_tt_pred.numpy(), + lam, + d_tol, + maxit=100, + STR_iters=10, + l0_penalty=None, + normalize=2, + split=0.8, + print_best_tol=False, + uv_flag=True) + lambda_v2 = self.train_stridge( + Phi_v.numpy(), + V_tt_pred.numpy(), + lam, + d_tol, + maxit=100, + STR_iters=10, + l0_penalty=None, + normalize=2, + split=0.8, + print_best_tol=False, + uv_flag=False) + + return lambda_u2, lambda_v2 + + def train_stridge(self, R, Ut, lam, d_tol, maxit, STR_iters=10, l0_penalty=None, normalize=2, + split=0.8, print_best_tol=False, uv_flag=True): + """ + Inspired by Rudy, Samuel H., et al. "Data-driven discovery of partial differential equations." + Science Advances 3.4 (2017): e1602614. + """ + + # Split data into 80% training and 20% test, then search for the best tolderance. + np.random.seed(0) # for consistency + n, _ = R.shape + train = np.random.choice(n, int(n * split), replace=False) + test = [i for i in np.arange(n) if i not in train] + TestR = R[test, :] + TestY = Ut[test, :] + + # Set up the initial tolerance and l0 penalty + d_tol = float(d_tol) + tol = d_tol + + # Get the standard least squares estimator + if uv_flag: + w_best = self.lambda_u.numpy() + else: + w_best = self.lambda_v.numpy() + + err_f = np.mean((TestY - TestR.dot(w_best))**2) + + if uv_flag: + self.l0_penalty_0_u = 10 * err_f + l0_penalty = self.l0_penalty_0_u + else: + self.l0_penalty_0_v = 10 * err_f + l0_penalty = self.l0_penalty_0_v + + err_lambda = l0_penalty * np.count_nonzero(w_best) + err_best = err_f + err_lambda + tol_best = 0 + + # Now increase tolerance until test performance decreases + for iteration in range(maxit): + print(f"Iteration {iteration + 1}") + + # Get a set of coefficients and error + w = self.stridge(R, Ut, lam, STR_iters, tol, normalize=normalize, uv_flag=uv_flag) + err_f = np.mean((TestY - TestR.dot(w))**2) + + err_lambda = l0_penalty * np.count_nonzero(w) + err = err_f + err_lambda + + # Has the accuracy improved? + if err <= err_best: + err_best = err + w_best = w + tol_best = tol + tol = 1.2 * tol + + else: + tol = 0.8 * tol + + if print_best_tol: + print("Optimal tolerance:", tol_best) + + optimaltol_history = np.empty([0]) + optimaltol_history = np.append(optimaltol_history, tol_best) + + return np.real(w_best) + + def stridge(self, X0, y, lam, maxit, tol, normalize=2, uv_flag=True): + """stridge""" + n, d = X0.shape + X = np.zeros((n, d), dtype=np.float32) + # First normalize data + if normalize != 0: + Mreg = np.zeros((d, 1)) + for i in range(0, d): + Mreg[i] = 1.0 / (np.linalg.norm(X0[:, i], normalize)) + X[:, i] = Mreg[i] * X0[:, i] + else: + X = X0 + + # Get the standard ridge estimate + # Inherit w from previous training + if uv_flag: + w = self.lambda_u.numpy() / Mreg + else: + w = self.lambda_v.numpy() / Mreg + + num_relevant = d + biginds = np.where(abs(w) > tol)[0] + + # Threshold and continue + for j in range(maxit): + + # Figure out which items to cut out + smallinds = np.where(abs(w) < tol)[0] + new_biginds = [i for i in range(d) if i not in smallinds] + + # If nothing changes then stop + if num_relevant == len(new_biginds): + break + else: + num_relevant = len(new_biginds) + + # Also make sure we didn't just lose all the coefficients + if not new_biginds: + if j == 0: + if normalize != 0: + w = np.multiply(Mreg, w) + return w + return w + break + biginds = new_biginds + + # Otherwise get a new guess + w[smallinds] = 0 + + if lam != 0: + w[biginds] = np.linalg.lstsq(X[:, biginds].T.dot( + X[:, biginds]) + lam * np.eye(len(biginds)), X[:, biginds].T.dot(y))[0] + else: + w[biginds] = np.linalg.lstsq(X[:, biginds], y)[0] + + # Now that we have the sparsity pattern, use standard least squares to get w + if biginds != []: + w[biginds] = np.linalg.lstsq(X[:, biginds].T.dot( + X[:, biginds]) + lam * np.eye(len(biginds)), X[:, biginds].T.dot(y))[0] + + if normalize != 0: + w = np.multiply(Mreg, w) + return w + return w diff --git a/MindFlow/applications/research/ICNet/src/network_SKG.py b/MindFlow/applications/research/ICNet/src/network_SKG.py new file mode 100644 index 0000000000000000000000000000000000000000..88b2c8ba4d46c665eac8732c7cae96d4800b4f4b --- /dev/null +++ b/MindFlow/applications/research/ICNet/src/network_SKG.py @@ -0,0 +1,423 @@ +# Copyright 2023 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================ +# pylint: disable=C0103 +"""Network definitions""" +import numpy as np + +import mindspore as ms +from mindspore import nn, Tensor, ops, Parameter +from mindspore.common.api import jit +from mindspore.amp import DynamicLossScaler +from mindspore.ops.functional import norm +from src.derivatives import SecondOrderGrad, Grad + + +# the deep neural network +class NeuralNet(nn.Cell): + """NeuralNetwork""" + def __init__(self, layers, msfloat_type): + super(NeuralNet, self).__init__() + + # parameters + self.depth = len(layers) - 1 + + # set up activation + self.activation = nn.Tanh() + + layer_list = [] + for i in range(self.depth - 1): + layer_list.append(nn.Dense(layers[i], layers[i + 1]).to_float(msfloat_type)) + layer_list.append(self.activation.to_float(msfloat_type)) + + layer_list.append(nn.Dense(layers[-2], layers[-1]).to_float(msfloat_type)) + + # deploy layers + self.layers = nn.SequentialCell(layer_list).to_float(msfloat_type) + + @jit + def construct(self, x): + out = self.layers(x) + return out + + +class InvarianceConstrainedNN: + """InvarianceConstrainedNN""" + # Initialize the class + def __init__(self, X, u, X_f, layers, batchno, use_npu, msfloat_type): + + # The Size of Network + self.layers = layers + + self.use_npu = use_npu + + self.msfloat_type = msfloat_type + + # Initialize trainable parameters + self.lambda_u = Tensor(np.zeros([4, 1]), self.msfloat_type) + self.lambda_uxx = Tensor([0.0], self.msfloat_type) + self.lambda_uyy = Tensor([0.0], self.msfloat_type) + + self.lambda_u = Parameter(self.lambda_u, name="name_u", requires_grad=True) + self.lambda_uxx = Parameter(self.lambda_uxx, name="name_uxx", requires_grad=True) + self.lambda_uyy = Parameter(self.lambda_uyy, name="name_uyy", requires_grad=True) + + # Training data + self.x = Tensor(np.array(X[:, 0:1], np.float32), self.msfloat_type) + self.y = Tensor(np.array(X[:, 1:2], np.float32), self.msfloat_type) + self.t = Tensor(np.array(X[:, 2:3], np.float32), self.msfloat_type) + self.u = Tensor(np.array(u, np.float32), self.msfloat_type) + + # Collection data for physics + self.x_f = Tensor(np.array(X_f[:, 0:1], np.float32), self.msfloat_type) + self.y_f = Tensor(np.array(X_f[:, 1:2], np.float32), self.msfloat_type) + self.t_f = Tensor(np.array(X_f[:, 2:3], np.float32), self.msfloat_type) + + self.batchno = batchno + self.batchsize_f = np.floor(self.x_f.shape[0] / self.batchno) + + # Initialize NNs---deep neural networks + self.dnn = NeuralNet(layers, self.msfloat_type) + + params = self.dnn.trainable_params() + params.append(self.lambda_u) + params.append(self.lambda_uxx) + params.append(self.lambda_uyy) + + # The function of Auto-differentiation + self.grad = Grad(self.dnn) + self.hessian_u_tt = SecondOrderGrad(self.dnn, 2, 2, output_idx=0) + self.hessian_u_xx = SecondOrderGrad(self.dnn, 0, 0, output_idx=0) + self.hessian_u_yy = SecondOrderGrad(self.dnn, 1, 1, output_idx=0) + + # The loss function + self.l2_loss = nn.MSELoss() + self.reduce_mean = ops.ReduceMean() + + # optimizers: using the same settings + self.loss_scaler = DynamicLossScaler(1024, 2, 100) + + def net_u(self, x, y, t): + H = ops.concat([x, y, t], 1) + Y = self.dnn(H) + return Y + + def net_f(self, x_f, y_f, t_f, N_f): + """ The minspore autograd version of calculating residual """ + U = self.net_u(x_f, y_f, t_f) + u = U[:, 0:1] + + data = ops.concat([x_f, y_f, t_f], 1) + + u_tt = self.hessian_u_tt(data) + u_xx = self.hessian_u_xx(data) + u_yy = self.hessian_u_yy(data) + + lib_fun = [ + Tensor( + np.ones( + (int(N_f), + 1), + dtype=np.float32), + self.msfloat_type), + u, + u * u, + u * u * u] + + f_u = u_tt + self.lambda_uxx * u_xx + self.lambda_uyy * u_yy + + for i in range(len(lib_fun)): + f_u = f_u - lib_fun[i] * self.lambda_u[i:i + 1, 0:1] + + return f_u + + def loss_fn(self, x, y, t, x_f, y_f, t_f, u): + """loss_fn""" + U = self.net_u(x, y, t) + u_pred = U[:, 0:1] + + f_u_pred = self.net_f(x_f, y_f, t_f, self.batchsize_f) + + loss_u = self.l2_loss(u_pred, u) + + loss_f_u = self.l2_loss(f_u_pred, ms.ops.zeros_like(f_u_pred)) + + lambda_u = self.lambda_u + loss_lambda_u = 1e-7 * norm(lambda_u, ord=1) + + loss = loss_u + loss_f_u + loss_lambda_u + if self.use_npu: + loss = self.loss_scaler.scale(loss) + return loss, loss_u, loss_f_u, loss_lambda_u + + +class InvarianceConstrainedNN_STRdige: + """InvarianceConstrainedNN_STRdige""" + # Initialize the class + def __init__(self, X, u, X_f, layers, batchno, lambda_u, lambda_uxx, + lambda_uyy, load_params, second_path, msfloat_type): + + # The Size of Network + self.layers = layers + + self.msfloat_type = msfloat_type + + # Initialize trainable parameters + self.lambda_u = Tensor(lambda_u, self.msfloat_type) + self.lambda_uxx = Tensor(lambda_uxx, self.msfloat_type) + self.lambda_uyy = Tensor(lambda_uyy, self.msfloat_type) + + self.lambda_u = Parameter(self.lambda_u, name="name_u", requires_grad=True) + self.lambda_uxx = Parameter(self.lambda_uxx, name="name_uxx", requires_grad=True) + self.lambda_uyy = Parameter(self.lambda_uyy, name="name_uyy", requires_grad=True) + + # Training data + self.x = Tensor(np.array(X[:, 0:1], np.float32), self.msfloat_type) + self.y = Tensor(np.array(X[:, 1:2], np.float32), self.msfloat_type) + self.t = Tensor(np.array(X[:, 2:3], np.float32), self.msfloat_type) + self.u = Tensor(np.array(u, np.float32), self.msfloat_type) + + # Collection data for physics + self.x_f = Tensor(np.array(X_f[:, 0:1], np.float32), self.msfloat_type) + self.y_f = Tensor(np.array(X_f[:, 1:2], np.float32), self.msfloat_type) + self.t_f = Tensor(np.array(X_f[:, 2:3], np.float32), self.msfloat_type) + + self.batchno = batchno + self.batchsize_f = np.floor(self.x_f.shape[0] / self.batchno) + + # Initialize NNs---deep neural networks + self.dnn = NeuralNet(layers, self.msfloat_type) + if load_params: + params_dict = ms.load_checkpoint(f'model/{second_path}/model.ckpt') + ms.load_param_into_net(self.dnn, params_dict) + + params = self.dnn.trainable_params() + params.append(self.lambda_u) + params.append(self.lambda_uxx) + params.append(self.lambda_uyy) + + # The function of Auto-differentiation + self.grad = Grad(self.dnn) + self.hessian_u_tt = SecondOrderGrad(self.dnn, 2, 2, output_idx=0) + self.hessian_u_xx = SecondOrderGrad(self.dnn, 0, 0, output_idx=0) + self.hessian_u_yy = SecondOrderGrad(self.dnn, 1, 1, output_idx=0) + + # The loss function + self.l2_loss = nn.MSELoss() + self.reduce_mean = ops.ReduceMean() + + # optimizers: using the same settings + self.optimizer_Adam = nn.Adam(params, learning_rate=1e-3) + self.loss_scaler = DynamicLossScaler(1024, 2, 100) + + def net_u(self, x, y, t): + H = ops.concat([x, y, t], 1) + u = self.dnn(H) + return u + + def net_f(self, x_f, y_f, t_f, N_f): + """ The minspore autograd version of calculating residual """ + u = self.net_u(x_f, y_f, t_f) + data = ops.concat([x_f, y_f, t_f], 1) + + u_tt = self.hessian_u_tt(data) + u_xx = self.hessian_u_xx(data) + u_yy = self.hessian_u_yy(data) + + f_u = u_tt + self.lambda_uxx * u_xx + self.lambda_uyy * u_yy + Phi = ops.cat((Tensor(np.ones((int(N_f), 1), dtype=np.float32), self.msfloat_type), + u, u * u, u * u * u), 1) + + lib_fun = [ + Tensor( + np.ones( + (int(N_f), + 1), + dtype=np.float32), + self.msfloat_type), + u, + u * u, + u * u * u] + for i in range(len(lib_fun)): + f_u = f_u - lib_fun[i] * self.lambda_u[i:i + 1, 0:1] + + return f_u, Phi, u_tt, u_xx, u_yy + + def loss_fn(self, x, y, t, x_f, y_f, t_f, u): + """loss_fn""" + u_pred = self.net_u(x, y, t) + f_u_pred, _, _, _, _ = self.net_f(x_f, y_f, t_f, self.batchsize_f) + + loss_u = self.l2_loss(u_pred, u) + loss_f_u = self.l2_loss(f_u_pred, ops.zeros_like(f_u_pred)) + + lambda_u = self.lambda_u + loss_lambda_u = 1e-7 * norm(lambda_u, ord=1) + + loss = loss_u + loss_f_u + loss_lambda_u + return loss, loss_u, loss_f_u, loss_lambda_u + + def call_trainstridge(self, lam, d_tol): + """call_trainstridge""" + + _, Phi_u, u_tt, u_xx, u_yy = self.net_f(self.x_f, self.y_f, self.t_f, self.batchsize_f) + + U_tt_pred = u_tt + self.lambda_uxx * u_xx + self.lambda_uyy * u_yy + + lambda_u2 = self.train_stridge( + Phi_u.numpy(), + U_tt_pred.numpy(), + lam, + d_tol, + maxit=100, + STR_iters=10, + l0_penalty=None, + normalize=2, + split=0.8, + print_best_tol=False, + uv_flag=True) + + return lambda_u2 + + def train_stridge(self, R, Ut, lam, d_tol, maxit, STR_iters=10, l0_penalty=None, normalize=2, + split=0.8, print_best_tol=False, uv_flag=True): + """ + Inspired by Rudy, Samuel H., et al. "Data-driven discovery of partial differential equations." + Science Advances 3.4 (2017): e1602614. + """ + + # Split data into 80% training and 20% test, then search for the best tolderance. + np.random.seed(0) # for consistency + n, _ = R.shape + train = np.random.choice(n, int(n * split), replace=False) + test = [i for i in np.arange(n) if i not in train] + TestR = R[test, :] + TestY = Ut[test, :] + + # Set up the initial tolerance and l0 penalty + d_tol = float(d_tol) + tol = d_tol + + # Get the standard least squares estimator + if uv_flag: + w_best = self.lambda_u.numpy() # self.sess.run(self.lambda_u) + else: + w_best = self.lambda_u.numpy() # self.sess.run(self.lambda_v) + + err_f = np.mean((TestY - TestR.dot(w_best))**2) + + if uv_flag: + self.l0_penalty_0_u = 10 * err_f + l0_penalty = self.l0_penalty_0_u + else: + self.l0_penalty_0_v = 10 * err_f + l0_penalty = self.l0_penalty_0_v + + err_lambda = l0_penalty * np.count_nonzero(w_best) + err_best = err_f + err_lambda + tol_best = 0 + + # Now increase tolerance until test performance decreases + for iteration in range(maxit): + print(f"Iteration {iteration + 1}") + + # Get a set of coefficients and error + w = self.stridge(R, Ut, lam, STR_iters, tol, normalize=normalize, uv_flag=uv_flag) + err_f = np.mean((TestY - TestR.dot(w))**2) + + err_lambda = l0_penalty * np.count_nonzero(w) + err = err_f + err_lambda + + # Has the accuracy improved? + if err <= err_best: + err_best = err + w_best = w + tol_best = tol + tol = 1.2 * tol + + else: + tol = 0.8 * tol + + if print_best_tol: + print("Optimal tolerance:", tol_best) + + optimaltol_history = np.empty([0]) + optimaltol_history = np.append(optimaltol_history, tol_best) + + return np.real(w_best) + + def stridge(self, X0, y, lam, maxit, tol, normalize=2, uv_flag=True): + """stridge""" + n, d = X0.shape + X = np.zeros((n, d), dtype=np.float32) + # First normalize data + if normalize != 0: + Mreg = np.zeros((d, 1)) + for i in range(0, d): + Mreg[i] = 1.0 / (np.linalg.norm(X0[:, i], normalize)) + X[:, i] = Mreg[i] * X0[:, i] + else: + X = X0 + + # Get the standard ridge estimate + # Inherit w from previous training + if uv_flag: + w = self.lambda_u.numpy() / Mreg + else: + w = self.lambda_u.numpy() / Mreg + + num_relevant = d + biginds = np.where(abs(w) > tol)[0] + + # Threshold and continue + for j in range(maxit): + + # Figure out which items to cut out + smallinds = np.where(abs(w) < tol)[0] + new_biginds = [i for i in range(d) if i not in smallinds] + + # If nothing changes then stop + if num_relevant == len(new_biginds): + break + else: + num_relevant = len(new_biginds) + + # Also make sure we didn't just lose all the coefficients + if not new_biginds: + if j == 0: + if normalize != 0: + w = np.multiply(Mreg, w) + return w + return w + break + biginds = new_biginds + + # Otherwise get a new guess + w[smallinds] = 0 + + if lam != 0: + w[biginds] = np.linalg.lstsq(X[:, biginds].T.dot( + X[:, biginds]) + lam * np.eye(len(biginds)), X[:, biginds].T.dot(y))[0] + else: + w[biginds] = np.linalg.lstsq(X[:, biginds], y)[0] + + # Now that we have the sparsity pattern, use standard least squares to get w + if biginds != []: + w[biginds] = np.linalg.lstsq(X[:, biginds].T.dot( + X[:, biginds]) + lam * np.eye(len(biginds)), X[:, biginds].T.dot(y))[0] + + if normalize != 0: + w = np.multiply(Mreg, w) + return w + return w diff --git a/MindFlow/applications/research/ICNet/src/network_burgers.py b/MindFlow/applications/research/ICNet/src/network_burgers.py new file mode 100644 index 0000000000000000000000000000000000000000..a64ceb954bc0f535f6604ebf54f3e0844267f36c --- /dev/null +++ b/MindFlow/applications/research/ICNet/src/network_burgers.py @@ -0,0 +1,503 @@ +# Copyright 2023 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================ +# pylint: disable=C0103 +"""Network definitions""" +import numpy as np + +import mindspore as ms +from mindspore import nn, Tensor, ops, Parameter +from mindspore.common.api import jit +from mindspore.amp import DynamicLossScaler +from mindspore.ops.functional import norm +from src.derivatives import SecondOrderGrad, Grad + + +# the deep neural network +class NeuralNet(nn.Cell): + """NeuralNetwork""" + def __init__(self, layers, msfloat_type): + super(NeuralNet, self).__init__() + + # parameters + self.depth = len(layers) - 1 + + # set up activation + self.activation = nn.Tanh() + + layer_list = [] + for i in range(self.depth - 1): + layer_list.append(nn.Dense(layers[i], layers[i + 1]).to_float(msfloat_type)) + layer_list.append(self.activation.to_float(msfloat_type)) + + layer_list.append(nn.Dense(layers[-2], layers[-1]).to_float(msfloat_type)) + + # deploy layers + self.layers = nn.SequentialCell(layer_list).to_float(msfloat_type) + + @jit + def construct(self, x): + out = self.layers(x) + return out + + +class InvarianceConstrainedNN: + """InvarianceConstrainedNN""" + # Initialize the class + def __init__(self, X, u, v, X_f, layers, batchno, use_npu, msfloat_type): + + # The Size of Network + self.layers = layers + + self.use_npu = use_npu + + self.msfloat_type = msfloat_type + + # Initialize trainable parameters + self.lambda_u = Tensor(np.zeros([11, 1]), self.msfloat_type) + self.lambda_v = Tensor(np.zeros([11, 1]), self.msfloat_type) + self.lambda_uux = Tensor([0.0], self.msfloat_type) + self.lambda_vuy = Tensor([0.0], self.msfloat_type) + self.lambda_uvx = Tensor([0.0], self.msfloat_type) + self.lambda_vvy = Tensor([0.0], self.msfloat_type) + + self.lambda_u = Parameter(self.lambda_u, name="name_u", requires_grad=True) + self.lambda_v = Parameter(self.lambda_v, name="name_v", requires_grad=True) + + self.lambda_uux = Parameter(self.lambda_uux, name="name_uux", requires_grad=True) + self.lambda_vuy = Parameter(self.lambda_vuy, name="name_vuy", requires_grad=True) + self.lambda_uvx = Parameter(self.lambda_uvx, name="name_uvx", requires_grad=True) + self.lambda_vvy = Parameter(self.lambda_vvy, name="name_vvy", requires_grad=True) + + # Training data + self.x = Tensor(np.array(X[:, 0:1], np.float32), self.msfloat_type) + self.y = Tensor(np.array(X[:, 1:2], np.float32), self.msfloat_type) + self.t = Tensor(np.array(X[:, 2:3], np.float32), self.msfloat_type) + self.u = Tensor(np.array(u, np.float32), self.msfloat_type) + self.v = Tensor(np.array(v, np.float32), self.msfloat_type) + + # Collection data for physics + self.x_f = Tensor(np.array(X_f[:, 0:1], np.float32), self.msfloat_type) + self.y_f = Tensor(np.array(X_f[:, 1:2], np.float32), self.msfloat_type) + self.t_f = Tensor(np.array(X_f[:, 2:3], np.float32), self.msfloat_type) + + self.batchno = batchno + self.batchsize_f = np.floor(self.x_f.shape[0] / self.batchno) + + # Initialize NNs---deep neural networks + self.dnn = NeuralNet(layers, self.msfloat_type) + + params = self.dnn.trainable_params() + params.append(self.lambda_u) + params.append(self.lambda_v) + params.append(self.lambda_uux) + params.append(self.lambda_vuy) + params.append(self.lambda_uvx) + params.append(self.lambda_vvy) + + # The function of Auto-differentiation + self.grad = Grad(self.dnn) + self.hessian_u_xx = SecondOrderGrad(self.dnn, 0, 0, output_idx=0) + self.hessian_u_xy = SecondOrderGrad(self.dnn, 0, 1, output_idx=0) + self.hessian_u_yy = SecondOrderGrad(self.dnn, 1, 1, output_idx=0) + + self.hessian_v_xx = SecondOrderGrad(self.dnn, 0, 0, output_idx=1) + self.hessian_v_xy = SecondOrderGrad(self.dnn, 0, 1, output_idx=1) + self.hessian_v_yy = SecondOrderGrad(self.dnn, 1, 1, output_idx=1) + + # The loss function + self.l2_loss = nn.MSELoss() + self.reduce_mean = ops.ReduceMean() + + self.loss_scaler = DynamicLossScaler(1024, 2, 100) + + def net_u(self, x, y, t): + H = ms.ops.concat([x, y, t], 1) + Y = self.dnn(H) + return Y + + def net_f(self, x_f, y_f, t_f, N_f): + """ The minspore autograd version of calculating residual """ + U = self.net_u(x_f, y_f, t_f) + u = U[:, 0:1] + v = U[:, 1:2] + + data = ms.ops.concat([x_f, y_f, t_f], 1) + + u_x = self.grad(data, 0, 0, U) + u_y = self.grad(data, 1, 0, U) + u_t = self.grad(data, 2, 0, U) + + u_xx = self.hessian_u_xx(data) + u_xy = self.hessian_u_xy(data) + u_yy = self.hessian_u_yy(data) + + v_x = self.grad(data, 0, 1, U) + v_y = self.grad(data, 1, 1, U) + v_t = self.grad(data, 2, 1, U) + + v_xx = self.hessian_v_xx(data) + v_xy = self.hessian_v_xy(data) + v_yy = self.hessian_v_yy(data) + + lib_fun = [ + Tensor( + np.ones( + (int(N_f), + 1), + dtype=np.float32), + self.msfloat_type), + u_x, + u_xx, + u_y, + u_yy, + u_xy, + v_x, + v_xx, + v_y, + v_yy, + v_xy] + + f_u = u_t + self.lambda_uux * u * u_x + self.lambda_vuy * v * u_y + f_v = v_t + self.lambda_uvx * u * v_x + self.lambda_vvy * v * v_y + + for i in range(len(lib_fun)): + f_u = f_u - lib_fun[i] * self.lambda_u[i:i + 1, 0:1] + f_v = f_v - lib_fun[i] * self.lambda_v[i:i + 1, 0:1] + + return f_u, f_v + + def loss_fn(self, x, y, t, x_f, y_f, t_f, u, v): + """loss_fn""" + U = self.net_u(x, y, t) + u_pred = U[:, 0:1] + v_pred = U[:, 1:2] + + f_u_pred, f_v_pred = self.net_f(x_f, y_f, t_f, self.batchsize_f) + + loss_u = self.l2_loss(u_pred, u) + loss_v = self.l2_loss(v_pred, v) + + loss_f_u = self.l2_loss(f_u_pred, ops.zeros_like(f_u_pred)) + loss_f_v = self.l2_loss(f_v_pred, ops.zeros_like(f_v_pred)) + + lambda_u = self.lambda_u + lambda_v = self.lambda_v + loss_lambda_u = 1e-7 * norm(lambda_u, ord=1) + loss_lambda_v = 1e-7 * norm(lambda_v, ord=1) + + loss = loss_u + loss_v + loss_f_u + loss_f_v + loss_lambda_u + loss_lambda_v + + if self.use_npu: + loss = self.loss_scaler.scale(loss) + + return loss, loss_u, loss_v, loss_f_u, loss_f_v, loss_lambda_u, loss_lambda_v + + +class InvarianceConstrainedNN_STRdige: + """"InvarianceConstrainedNN_STRdige""" + # Initialize the class + def __init__(self, X, u, v, X_f, layers, batchno, lambda_u, lambda_v, lambda_uux, + lambda_vuy, lambda_uvx, lambda_vvy, load_params, second_path, msfloat_type): + + self.msfloat_type = msfloat_type + + # Initialize trainable parameters + self.lambda_u = Tensor(lambda_u, self.msfloat_type) + self.lambda_v = Tensor(lambda_v, self.msfloat_type) + self.lambda_uux = Tensor(lambda_uux, self.msfloat_type) + self.lambda_vuy = Tensor(lambda_vuy, self.msfloat_type) + self.lambda_uvx = Tensor(lambda_uvx, self.msfloat_type) + self.lambda_vvy = Tensor(lambda_vvy, self.msfloat_type) + + self.lambda_u = Parameter(self.lambda_u, name="name_u", requires_grad=True) + self.lambda_v = Parameter(self.lambda_v, name="name_v", requires_grad=True) + + self.lambda_uux = Parameter(self.lambda_uux, name="name_uux", requires_grad=True) + self.lambda_vuy = Parameter(self.lambda_vuy, name="name_vuy", requires_grad=True) + self.lambda_uvx = Parameter(self.lambda_uvx, name="name_uvx", requires_grad=True) + self.lambda_vvy = Parameter(self.lambda_vvy, name="name_vvy", requires_grad=True) + + # Training data + self.x = Tensor(np.array(X[:, 0:1], np.float32), self.msfloat_type) + self.y = Tensor(np.array(X[:, 1:2], np.float32), self.msfloat_type) + self.t = Tensor(np.array(X[:, 2:3], np.float32), self.msfloat_type) + self.u = Tensor(np.array(u, np.float32), self.msfloat_type) + self.v = Tensor(np.array(v, np.float32), self.msfloat_type) + + # Collection data for physics + self.x_f = Tensor(np.array(X_f[:, 0:1], np.float32), self.msfloat_type) + self.y_f = Tensor(np.array(X_f[:, 1:2], np.float32), self.msfloat_type) + self.t_f = Tensor(np.array(X_f[:, 2:3], np.float32), self.msfloat_type) + + self.batchno = batchno + self.batchsize_f = np.floor(self.x_f.shape[0] / self.batchno) + + # Initialize NNs---deep neural networks + self.dnn = NeuralNet(layers, self.msfloat_type) + if load_params: + params_dict = ms.load_checkpoint(f'model/{second_path}/model.ckpt') + ms.load_param_into_net(self.dnn, params_dict) + + params = self.dnn.trainable_params() + params.append(self.lambda_u) + params.append(self.lambda_v) + params.append(self.lambda_uux) + params.append(self.lambda_vuy) + params.append(self.lambda_uvx) + params.append(self.lambda_vvy) + + # The function of Auto-differentiation + self.grad = Grad(self.dnn) + self.hessian_u_xx = SecondOrderGrad(self.dnn, 0, 0, output_idx=0) + self.hessian_u_xy = SecondOrderGrad(self.dnn, 0, 1, output_idx=0) + self.hessian_u_yy = SecondOrderGrad(self.dnn, 1, 1, output_idx=0) + + self.hessian_v_xx = SecondOrderGrad(self.dnn, 0, 0, output_idx=1) + self.hessian_v_xy = SecondOrderGrad(self.dnn, 0, 1, output_idx=1) + self.hessian_v_yy = SecondOrderGrad(self.dnn, 1, 1, output_idx=1) + + # The loss function + self.l2_loss = nn.MSELoss() + self.reduce_mean = ops.ReduceMean() + + # optimizers: using the same settings + self.loss_scaler = DynamicLossScaler(1024, 2, 100) + + def net_u(self, x, y, t): + H = ops.concat([x, y, t], 1) + Y = self.dnn(H) + return Y + + def net_f(self, x_f, y_f, t_f, N_f): + """ The minspore autograd version of calculating residual """ + U = self.net_u(x_f, y_f, t_f) + u = U[:, 0:1] + v = U[:, 1:2] + + data = ops.concat([x_f, y_f, t_f], 1) + + u_x = self.grad(data, 0, 0, U) + u_y = self.grad(data, 1, 0, U) + u_t = self.grad(data, 2, 0, U) + + u_xx = self.hessian_u_xx(data) + u_xy = self.hessian_u_xy(data) + u_yy = self.hessian_u_yy(data) + + v_x = self.grad(data, 0, 1, U) + v_y = self.grad(data, 1, 1, U) + v_t = self.grad(data, 2, 1, U) + + v_xx = self.hessian_v_xx(data) + v_xy = self.hessian_v_xy(data) + v_yy = self.hessian_v_yy(data) + + Phi_u = ops.cat((Tensor(np.ones((int(N_f), 1), dtype=np.float32), self.msfloat_type), + u_x, u_xx, u_y, u_yy, u_xy, v_x, v_xx, v_y, v_yy, v_xy), 1) + Phi_v = ops.cat((Tensor(np.ones((int(N_f), 1), dtype=np.float32), self.msfloat_type), + u_x, u_xx, u_y, u_yy, u_xy, v_x, v_xx, v_y, v_yy, v_xy), 1) + + f_u = u_t + self.lambda_uux * u * u_x + self.lambda_vuy * v * u_y + f_v = v_t + self.lambda_uvx * u * v_x + self.lambda_vvy * v * v_y + + return f_u, f_v, Phi_u, Phi_v + + def loss_fn(self, x, y, t, x_f, y_f, t_f, u, v): + """loss_fn""" + U = self.net_u(x, y, t) + u_pred = U[:, 0:1] + v_pred = U[:, 1:2] + + f_u_pred, f_v_pred = self.net_f(x_f, y_f, t_f, self.batchsize_f) + + loss_u = self.l2_loss(u_pred, u) + loss_v = self.l2_loss(v_pred, v) + + loss_f_u = self.l2_loss(f_u_pred, ops.zeros_like(f_u_pred)) + loss_f_v = self.l2_loss(f_v_pred, ops.zeros_like(f_v_pred)) + + lambda_u = self.lambda_u + lambda_v = self.lambda_v + loss_lambda_u = 1e-7 * norm(lambda_u, ord=1) + loss_lambda_v = 1e-7 * norm(lambda_v, ord=1) + + loss = loss_u + loss_v + loss_f_u + loss_f_v + loss_lambda_u + loss_lambda_v + + return loss, loss_u, loss_v, loss_f_u, loss_f_v, loss_lambda_u, loss_lambda_v + + def call_trainstridge(self, lam, d_tol): + """call_trainstridge""" + U_t_pred, V_t_pred, Phi_u, Phi_v = self.net_f( + self.x_f, self.y_f, self.t_f, self.batchsize_f) + + lambda_u2 = self.train_stridge( + Phi_u.numpy(), + U_t_pred.numpy(), + lam, + d_tol, + maxit=100, + STR_iters=10, + l0_penalty=None, + normalize=2, + split=0.8, + print_best_tol=False, + uv_flag=True) + lambda_v2 = self.train_stridge( + Phi_v.numpy(), + V_t_pred.numpy(), + lam, + d_tol, + maxit=100, + STR_iters=10, + l0_penalty=None, + normalize=2, + split=0.8, + print_best_tol=False, + uv_flag=False) + + return lambda_u2, lambda_v2 + + def train_stridge(self, R, Ut, lam, d_tol, maxit, STR_iters=10, l0_penalty=None, normalize=2, + split=0.8, print_best_tol=False, uv_flag=True): + """ + Inspired by Rudy, Samuel H., et al. "Data-driven discovery of partial differential equations." + Science Advances 3.4 (2017): e1602614. + """ + + # Split data into 80% training and 20% test, then search for the best tolderance. + np.random.seed(0) # for consistency + n, _ = R.shape + train = np.random.choice(n, int(n * split), replace=False) + test = [i for i in np.arange(n) if i not in train] + TestR = R[test, :] + TestY = Ut[test, :] + + # Set up the initial tolerance and l0 penalty + d_tol = float(d_tol) + tol = d_tol + + # Get the standard least squares estimator + if uv_flag: + w_best = self.lambda_u.numpy() + else: + w_best = self.lambda_v.numpy() + + err_f = np.mean((TestY - TestR.dot(w_best))**2) + + # Get the standard least squares estimator + if uv_flag: + self.l0_penalty_0_u = 10 * err_f + l0_penalty = self.l0_penalty_0_u + else: + self.l0_penalty_0_v = 10 * err_f + l0_penalty = self.l0_penalty_0_v + + err_lambda = l0_penalty * np.count_nonzero(w_best) + err_best = err_f + err_lambda + tol_best = 0 + + # Now increase tolerance until test performance decreases + for iteration in range(maxit): + print(f"Iteration {iteration + 1}") + + # Get a set of coefficients and error + w = self.stridge(R, Ut, lam, STR_iters, tol, normalize=normalize, uv_flag=uv_flag) + err_f = np.mean((TestY - TestR.dot(w))**2) + + err_lambda = l0_penalty * np.count_nonzero(w) + err = err_f + err_lambda + + # Has the accuracy improved? + if err <= err_best: + err_best = err + w_best = w + tol_best = tol + tol = 1.2 * tol + + else: + tol = 0.8 * tol + + if print_best_tol: + print("Optimal tolerance:", tol_best) + + optimaltol_history = np.empty([0]) + optimaltol_history = np.append(optimaltol_history, tol_best) + + return np.real(w_best) + + def stridge(self, X0, y, lam, maxit, tol, normalize=2, uv_flag=True): + """stridge""" + n, d = X0.shape + X = np.zeros((n, d), dtype=np.float32) + # First normalize data + if normalize != 0: + Mreg = np.zeros((d, 1)) + for i in range(0, d): + Mreg[i] = 1.0 / (np.linalg.norm(X0[:, i], normalize)) + X[:, i] = Mreg[i] * X0[:, i] + else: + X = X0 + + # Get the standard ridge estimate + # Inherit w from previous training + if uv_flag: + w = self.lambda_u.numpy() / Mreg + else: + w = self.lambda_v.numpy() / Mreg + + num_relevant = d + biginds = np.where(abs(w) > tol)[0] + + # Threshold and continue + for j in range(maxit): + + # Figure out which items to cut out + smallinds = np.where(abs(w) < tol)[0] + new_biginds = [i for i in range(d) if i not in smallinds] + + # If nothing changes then stop + if num_relevant == len(new_biginds): + break + else: + num_relevant = len(new_biginds) + + # Also make sure we didn't just lose all the coefficients + if not new_biginds: + if j == 0: + if normalize != 0: + w = np.multiply(Mreg, w) + return w + return w + break + biginds = new_biginds + + # Otherwise get a new guess + w[smallinds] = 0 + + if lam != 0: + w[biginds] = np.linalg.lstsq(X[:, biginds].T.dot( + X[:, biginds]) + lam * np.eye(len(biginds)), X[:, biginds].T.dot(y))[0] + else: + w[biginds] = np.linalg.lstsq(X[:, biginds], y)[0] + + # Now that we have the sparsity pattern, use standard least squares to get w + if biginds != []: + w[biginds] = np.linalg.lstsq(X[:, biginds].T.dot( + X[:, biginds]) + lam * np.eye(len(biginds)), X[:, biginds].T.dot(y))[0] + + if normalize != 0: + w = np.multiply(Mreg, w) + return w + return w diff --git a/MindFlow/applications/research/ICNet/train_GI_1D_KS.py b/MindFlow/applications/research/ICNet/train_GI_1D_KS.py new file mode 100644 index 0000000000000000000000000000000000000000..654db6c9a4232100e2821c2d6d2facec23af3608 --- /dev/null +++ b/MindFlow/applications/research/ICNet/train_GI_1D_KS.py @@ -0,0 +1,209 @@ +# Copyright 2023 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================ +# pylint: disable=C0103 +"""ICNet train""" +import time +import argparse +import yaml +import numpy as np + +import mindspore as ms +from mindspore import set_seed, context, nn +from src.network import InvarianceConstrainedNN, InvarianceConstrainedNN_STRdige +from src.datasets import read_training_data, print_pde + + +# set seed +np.random.seed(123456) +set_seed(123456) + + +def train(model, NIter, lr): + """train""" + # Get the gradients function + params_train = model.dnn.trainable_params() + params_train.append(model.lambda_u) + params_train.append(model.lambda_uux) + + optimizer_Adam = nn.Adam(params_train, learning_rate=lr) + + grad_fn = ms.value_and_grad(model.loss_fn, None, optimizer_Adam.parameters, has_aux=True) + + model.dnn.set_train() + + start_time = time.time() + + for epoch in range(1, 1 + NIter): + (loss, loss_u, loss_f_u, loss_lambda_u), grads = grad_fn( + model.x, model.t, model.x_f, model.t_f, model.u) + + optimizer_Adam(grads) + + if epoch % 10 == 0: + elapsed = time.time() - start_time + print('It: %d, Loss: %.3e, loss_u: %.3e, loss_f: %.3e, loss_lambda: %.3e, Lambda_uux: %.3f, Lambda_uxx: %.3f, Lambda_uxxxx: %.3f, Time: %.2f' % + (epoch, loss.item(), loss_u.item(), loss_f_u.item(), loss_lambda_u.item(), + model.lambda_uux.item(), model.lambda_u[2].item(), model.lambda_u[4].item(), elapsed)) + + initial_size = 5 + + loss_history_Adam_Pretrain = np.empty([0]) + loss_u_history_Adam_Pretrain = np.empty([0]) + loss_f_u_history_Adam_Pretrain = np.empty([0]) + loss_lambda_u_history_Adam_Pretrain = np.empty([0]) + + lambda_u_history_Adam_Pretrain = np.zeros((initial_size, 1)) + lambda_uux_history_Adam_Pretrain = np.zeros((1, 1)) + + loss_history_Adam_Pretrain = np.append(loss_history_Adam_Pretrain, loss.numpy()) + lambda_u_history_Adam_Pretrain = np.append( + lambda_u_history_Adam_Pretrain, model.lambda_u.numpy(), axis=1) + loss_u_history_Adam_Pretrain = np.append( + loss_u_history_Adam_Pretrain, loss_u.numpy()) + loss_f_u_history_Adam_Pretrain = np.append( + loss_f_u_history_Adam_Pretrain, loss_f_u.numpy()) + loss_lambda_u_history_Adam_Pretrain = np.append( + loss_lambda_u_history_Adam_Pretrain, loss_lambda_u.numpy()) + + lambda_uux_new = np.array([model.lambda_uux.numpy()]) + lambda_uux_history_Adam_Pretrain = np.append( + lambda_uux_history_Adam_Pretrain, lambda_uux_new, axis=1) + + start_time = time.time() + np.save(f'Loss-Coe/{second_path}/loss_history_Adam_Pretrain', loss_history_Adam_Pretrain) + np.save( + f'Loss-Coe/{second_path}/loss_u_history_Adam_Pretrain', + loss_u_history_Adam_Pretrain) + np.save( + f'Loss-Coe/{second_path}/loss_f_u_history_Adam_Pretrain', + loss_f_u_history_Adam_Pretrain) + np.save( + f'Loss-Coe/{second_path}/loss_lambda_u_history_Adam_Pretrain', + loss_lambda_u_history_Adam_Pretrain) + + np.save( + f'Loss-Coe/{second_path}/lambda_u_history_Adam_Pretrain', + lambda_u_history_Adam_Pretrain) + np.save( + f'Loss-Coe/{second_path}/lambda_uux_history_Adam_Pretrain', + lambda_uux_history_Adam_Pretrain) + + +if __name__ == "__main__": + parser = argparse.ArgumentParser() + parser.add_argument("--config_file_path", type=str, default="./config/ICNet_KS.yaml") + args = parser.parse_known_args()[0] + + with open(args.config_file_path, 'r') as f: + config = yaml.safe_load(f) + + params = config.get('params', {}) + for key, value in params.items(): + setattr(args, key, value) + + model_name = args.model_name + case = args.case + device = args.device + device_id = args.device_id + network_size = args.network_size + learning_rate = args.learning_rate + epochs = args.epochs + BatchNo = args.BatchNo + load_params = args.load_params + second_path = args.second_path + description_ks = args.description_ks + lam = args.lam + d_tol = args.d_tol + + context.set_context( + mode=context.GRAPH_MODE, + save_graphs=False, + device_target=device, + device_id=device_id) + + use_npu = (device == 'Ascend') + + if use_npu: + msfloat_type = ms.float16 + else: + msfloat_type = ms.float32 + + X_u_train, u_train, X_f_train = read_training_data(args) + + model_pretrain = InvarianceConstrainedNN( + X_u_train, + u_train, + X_f_train, + network_size, + BatchNo, + use_npu, + msfloat_type) + for epoch_train, lr_value in zip(epochs, learning_rate): + train(model_pretrain, int(float(epoch_train)), lr_value) + ms.save_checkpoint(model_pretrain.dnn, f'model/model.ckpt') + + np.save(f'Loss-Coe/{second_path}/loss_history_Adam_Pretrain', loss_history_Adam_Pretrain) + np.save( + f'Loss-Coe/{second_path}/loss_u_history_Adam_Pretrain', + loss_u_history_Adam_Pretrain) + np.save( + f'Loss-Coe/{second_path}/loss_f_u_history_Adam_Pretrain', + loss_f_u_history_Adam_Pretrain) + np.save( + f'Loss-Coe/{second_path}/loss_lambda_u_history_Adam_Pretrain', + loss_lambda_u_history_Adam_Pretrain) + + np.save( + f'Loss-Coe/{second_path}/lambda_u_history_Adam_Pretrain', + lambda_u_history_Adam_Pretrain) + np.save( + f'Loss-Coe/{second_path}/lambda_uux_history_Adam_Pretrain', + lambda_uux_history_Adam_Pretrain) + + lambda_uux_value = model_pretrain.lambda_uux.numpy() + lambda_u_value = model_pretrain.lambda_u.numpy() + np.save(f'Loss-Coe/{second_path}/lambda_uux_value', lambda_uux_value) + np.save(f'Loss-Coe/{second_path}/lambda_u_value', lambda_u_value) + + if load_params: + lambda_u_value = np.load(f'Loss-Coe/{second_path}/lambda_u_value.npy') + lambda_uux_value = np.load(f'Loss-Coe/{second_path}/lambda_uux_value.npy') + model_ICCO = InvarianceConstrainedNN_STRdige( + X_u_train, + u_train, + X_f_train, + network_size, + BatchNo, + lambda_u_value, + lambda_uux_value, + load_params, + second_path, + msfloat_type) + else: + model_ICCO = InvarianceConstrainedNN_STRdige( + X_u_train, + u_train, + X_f_train, + network_size, + BatchNo, + lambda_u_value, + lambda_uux_value, + load_params, + second_path, + msfloat_type) + + lambda_u_STRidge = model_ICCO.call_trainstridge(lam, d_tol) + + print_pde(lambda_uux_value, lambda_u_STRidge, description_ks, ut='u_t') diff --git a/MindFlow/applications/research/ICNet/train_GI_2D_Burgers.py b/MindFlow/applications/research/ICNet/train_GI_2D_Burgers.py new file mode 100644 index 0000000000000000000000000000000000000000..4181f47f01f2f0c5fb1efb66674d6fdb668f4502 --- /dev/null +++ b/MindFlow/applications/research/ICNet/train_GI_2D_Burgers.py @@ -0,0 +1,217 @@ +# Copyright 2023 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================ +# pylint: disable=C0103 +"""ICNet train""" +import time +import argparse +import yaml +import numpy as np + +import mindspore as ms +from mindspore import set_seed, context, nn +from src.network_burgers import InvarianceConstrainedNN, InvarianceConstrainedNN_STRdige +from src.datasets_burgers import read_training_data, print_pde + + +# Set the seed. +np.random.seed(123456) +set_seed(123456) + + +def train(model, NIter, lr): + """train""" + # Get the gradients function + params_train = model.dnn.trainable_params() + params_train.append(model.lambda_u) + params_train.append(model.lambda_v) + params_train.append(model.lambda_uux) + params_train.append(model.lambda_vuy) + params_train.append(model.lambda_uvx) + params_train.append(model.lambda_vvy) + + optimizer_Adam = nn.Adam(params_train, learning_rate=lr) + + grad_fn = ms.value_and_grad(model.loss_fn, None, optimizer_Adam.parameters, has_aux=True) + + model.dnn.set_train() + + start_time = time.time() + + for epoch in range(1, 1 + NIter): + (loss, _, _, _, _, _, _), grads = grad_fn(model.x, model.y, model.t, + model.x_f, model.y_f, model.t_f, model.u, model.v) + optimizer_Adam(grads) + if epoch % 10 == 0: + elapsed = time.time() - start_time + print('It: %d, Loss: %.3e, lambda_uux: %.3f, lambda_vuy: %.3f, lambda_uxx: %.3f, lambda_uyy: %.3f,\ + lambda_uvx: %.3f, lambda_vvy: %.3f, lambda_vxx: %.3f, lambda_vyy: %.3f, Time: %.2f' % + (epoch, loss.item(), + model.lambda_uux.item(), model.lambda_vuy.item(), + model.lambda_u[2].item(), model.lambda_u[4].item(), + model.lambda_uvx.item(), model.lambda_vvy.item(), model.lambda_v[7].item(), model.lambda_v[9].item(), elapsed)) + + initial_size = 11 + loss_history_Adam_Pretrain = np.empty([0]) + lambda_u_history_Adam_Pretrain = np.zeros((initial_size, 1)) + lambda_v_history_Adam_Pretrain = np.zeros((initial_size, 1)) + lambda_uux_history_Adam_Pretrain = np.zeros((1, 1)) + lambda_vuy_history_Adam_Pretrain = np.zeros((1, 1)) + lambda_uvx_history_Adam_Pretrain = np.zeros((1, 1)) + lambda_vvy_history_Adam_Pretrain = np.zeros((1, 1)) + + loss_history_Adam_Pretrain = np.append(loss_history_Adam_Pretrain, loss.numpy()) + lambda_u_history_Adam_Pretrain = np.append( + lambda_u_history_Adam_Pretrain, model.lambda_u.numpy(), axis=1) + lambda_v_history_Adam_Pretrain = np.append( + lambda_v_history_Adam_Pretrain, model.lambda_v.numpy(), axis=1) + + lambda_uux_new = np.array([model.lambda_uux.numpy()]) + lambda_vuy_new = np.array([model.lambda_vuy.numpy()]) + lambda_uvx_new = np.array([model.lambda_uvx.numpy()]) + lambda_vvy_new = np.array([model.lambda_vvy.numpy()]) + + lambda_uux_history_Adam_Pretrain = np.append( + lambda_uux_history_Adam_Pretrain, lambda_uux_new, axis=1) + lambda_vuy_history_Adam_Pretrain = np.append( + lambda_vuy_history_Adam_Pretrain, lambda_vuy_new, axis=1) + lambda_uvx_history_Adam_Pretrain = np.append( + lambda_uvx_history_Adam_Pretrain, lambda_uvx_new, axis=1) + lambda_vvy_history_Adam_Pretrain = np.append( + lambda_vvy_history_Adam_Pretrain, lambda_vvy_new, axis=1) + + start_time = time.time() + np.save(f'Loss-Coe/{second_path}/loss_history_Adam_Pretrain', loss_history_Adam_Pretrain) + np.save( + f'Loss-Coe/{second_path}/lambda_u_history_Adam_Pretrain', + lambda_u_history_Adam_Pretrain) + np.save( + f'Loss-Coe/{second_path}/lambda_v_history_Adam_Pretrain', + lambda_v_history_Adam_Pretrain) + np.save( + f'Loss-Coe/{second_path}/lambda_uux_history_Adam_Pretrain', + lambda_uux_history_Adam_Pretrain) + np.save( + f'Loss-Coe/{second_path}/lambda_vuy_history_Adam_Pretrain', + lambda_vuy_history_Adam_Pretrain) + np.save( + f'Loss-Coe/{second_path}/lambda_uvx_history_Adam_Pretrain', + lambda_uvx_history_Adam_Pretrain) + np.save( + f'Loss-Coe/{second_path}/lambda_vvy_history_Adam_Pretrain', + lambda_vvy_history_Adam_Pretrain) + +if __name__ == "__main__": + parser = argparse.ArgumentParser() + parser.add_argument("--config_file_path", type=str, default="./config/ICNet_Burgers.yaml") + args = parser.parse_known_args()[0] + + with open(args.config_file_path, 'r') as f: + config = yaml.safe_load(f) + + params = config.get('params', {}) + for key, value in params.items(): + setattr(args, key, value) + + model_name = args.model_name + case = args.case + device = args.device + device_id = args.device_id + network_size = args.network_size + learning_rate = args.learning_rate + epochs = args.epochs + BatchNo = args.BatchNo + load_params = args.load_params + second_path = args.second_path + description_burgers_u = args.description_burgers_u + description_burgers_v = args.description_burgers_v + lam = args.lam + d_tol = args.d_tol + + context.set_context( + mode=context.GRAPH_MODE, + save_graphs=False, + device_target=device, + device_id=device_id) + + use_npu = (device == 'Ascend') + + if use_npu: + msfloat_type = ms.float16 + else: + msfloat_type = ms.float32 + + X_u_train, u_train, v_train, X_f_train = read_training_data(args) + + model_pretrain = InvarianceConstrainedNN( + X_u_train, + u_train, + v_train, + X_f_train, + network_size, + BatchNo, + use_npu, + msfloat_type) + for epoch_train, lr_train in zip(epochs, learning_rate): + train(model_pretrain, int(float(epoch_train)), lr_train) + ms.save_checkpoint(model_pretrain.dnn, f'model/model.ckpt') + + lambda_uux_value = model_pretrain.lambda_uux.numpy() + lambda_vuy_value = model_pretrain.lambda_vuy.numpy() + lambda_uvx_value = model_pretrain.lambda_uvx.numpy() + lambda_vvy_value = model_pretrain.lambda_vvy.numpy() + lambda_u_value = model_pretrain.lambda_u.numpy() + lambda_v_value = model_pretrain.lambda_v.numpy() + + # Save the last coefficients + np.save(f'Loss-Coe/{second_path}/lambda_uux_value', lambda_uux_value) + np.save(f'Loss-Coe/{second_path}/lambda_vuy_value', lambda_vuy_value) + np.save(f'Loss-Coe/{second_path}/lambda_uvx_value', lambda_uvx_value) + np.save(f'Loss-Coe/{second_path}/lambda_vvy_value', lambda_vvy_value) + + np.save(f'Loss-Coe/{second_path}/lambda_u_value', lambda_u_value) + np.save(f'Loss-Coe/{second_path}/lambda_v_value', lambda_v_value) + + if load_params: + lambda_u_value = np.load(f'Loss-Coe/{second_path}/lambda_u_value.npy') + lambda_v_value = np.load(f'Loss-Coe/{second_path}/lambda_v_value.npy') + lambda_uux_value = np.load(f'Loss-Coe/{second_path}/lambda_uux_value.npy') + lambda_vuy_value = np.load(f'Loss-Coe/{second_path}/lambda_vuy_value.npy') + lambda_uvx_value = np.load(f'Loss-Coe/{second_path}/lambda_uvx_value.npy') + lambda_vvy_value = np.load(f'Loss-Coe/{second_path}/lambda_vvy_value.npy') + model_ICCO = InvarianceConstrainedNN_STRdige(X_u_train, u_train, v_train, X_f_train, network_size, BatchNo, + lambda_u_value, lambda_v_value, lambda_uux_value, lambda_vuy_value, lambda_uvx_value, lambda_vvy_value, + load_params, second_path, msfloat_type,) + else: + model_ICCO = InvarianceConstrainedNN_STRdige(X_u_train, u_train, v_train, X_f_train, network_size, BatchNo, + lambda_u_value, lambda_v_value, lambda_uux_value, lambda_vuy_value, lambda_uvx_value, lambda_vvy_value, + load_params, second_path, msfloat_type,) + + lam = 10**-5 + d_tol = 5 + lambda_u_STRidge, lambda_v_STRidge = model_ICCO.call_trainstridge(lam, d_tol) + + print_pde( + lambda_uux_value, + lambda_vuy_value, + lambda_u_STRidge, + description_burgers_u, + ut='u_t') + + print_pde( + lambda_uvx_value, + lambda_vvy_value, + lambda_v_STRidge, + description_burgers_v, + ut='v_t') diff --git a/MindFlow/applications/research/ICNet/train_LI_2D_CKG.py b/MindFlow/applications/research/ICNet/train_LI_2D_CKG.py new file mode 100644 index 0000000000000000000000000000000000000000..29f01fcc70528b7a9e777d00b3a76a8c48897cf3 --- /dev/null +++ b/MindFlow/applications/research/ICNet/train_LI_2D_CKG.py @@ -0,0 +1,215 @@ +# Copyright 2023 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================ +# pylint: disable=C0103 +"""ICNet train""" +import time +import argparse +import yaml +import numpy as np + +import mindspore as ms +from mindspore import set_seed, context, nn +from ChenChao.MindSpore.ICNet.ICNet.ICNet_PR.PR_003.src.network_ckg import InvarianceConstrainedNN, InvarianceConstrainedNN_STRdige +from src.datasets_CKG import read_training_data, print_pde + + +# Set seed +np.random.seed(123456) +set_seed(123456) + + +def train(model, NIter, lr): + """train""" + # Get the gradients function + params_train = model.dnn.trainable_params() + params_train.append(model.lambda_u) + params_train.append(model.lambda_v) + + params_train.append(model.lambda_uxx) + params_train.append(model.lambda_uyy) + params_train.append(model.lambda_vxx) + params_train.append(model.lambda_vyy) + + optimizer_Adam = nn.Adam(params_train, learning_rate=lr) + + grad_fn = ms.value_and_grad(model.loss_fn, None, optimizer_Adam.parameters, has_aux=True) + + model.dnn.set_train() + + start_time = time.time() + + for epoch in range(1, 1 + NIter): + (loss, _, _, _, _, _, _), grads = grad_fn(model.x, model.y, model.t, + model.x_f, model.y_f, model.t_f, model.u, model.v) + optimizer_Adam(grads) + + if epoch % 10 == 0: + elapsed = time.time() - start_time + print('It: %d, Loss: %.3e, lambda_u1: %.3f, lambda_u3: %.3f, lambda_uv2: %.3f, lambda_uxx: %.3f, lambda_uyy: %.3f,\ + lambda_v1: %.3f, lambda_v3: %.3f, lambda_vu2: %.3f, lambda_vxx: %.3f, lambda_vyy: %.3f, Time: %.2f' % + (epoch, loss.item(), + model.lambda_u[1].item(), model.lambda_u[3].item(), model.lambda_u[5].item(), + model.lambda_uxx.item(), model.lambda_uyy.item(), + model.lambda_v[6].item(), model.lambda_v[8].item(), model.lambda_v[9].item(), model.lambda_vxx.item(), model.lambda_vyy.item(), elapsed)) + + initial_size = 10 + loss_history_Adam_Pretrain = np.empty([0]) + lambda_u_history_Adam_Pretrain = np.zeros((initial_size, 1)) + lambda_v_history_Adam_Pretrain = np.zeros((initial_size, 1)) + lambda_uxx_history_Adam_Pretrain = np.zeros((1, 1)) + lambda_uyy_history_Adam_Pretrain = np.zeros((1, 1)) + lambda_vxx_history_Adam_Pretrain = np.zeros((1, 1)) + lambda_vyy_history_Adam_Pretrain = np.zeros((1, 1)) + + loss_history_Adam_Pretrain = np.append(loss_history_Adam_Pretrain, loss.numpy()) + lambda_u_history_Adam_Pretrain = np.append( + lambda_u_history_Adam_Pretrain, model.lambda_u.numpy(), axis=1) + lambda_v_history_Adam_Pretrain = np.append( + lambda_v_history_Adam_Pretrain, model.lambda_v.numpy(), axis=1) + lambda_uxx_new = np.array([model.lambda_uxx.numpy()]) + lambda_uyy_new = np.array([model.lambda_uyy.numpy()]) + lambda_vxx_new = np.array([model.lambda_vxx.numpy()]) + lambda_vyy_new = np.array([model.lambda_vyy.numpy()]) + lambda_uxx_history_Adam_Pretrain = np.append( + lambda_uxx_history_Adam_Pretrain, lambda_uxx_new, axis=1) + lambda_uyy_history_Adam_Pretrain = np.append( + lambda_uyy_history_Adam_Pretrain, lambda_uyy_new, axis=1) + lambda_vxx_history_Adam_Pretrain = np.append( + lambda_vxx_history_Adam_Pretrain, lambda_vxx_new, axis=1) + lambda_vyy_history_Adam_Pretrain = np.append( + lambda_vyy_history_Adam_Pretrain, lambda_vyy_new, axis=1) + + start_time = time.time() + + np.save(f'Loss-Coe/{second_path}/loss_history_Adam_Pretrain', loss_history_Adam_Pretrain) + np.save( + f'Loss-Coe/{second_path}/lambda_u_history_Adam_Pretrain', + lambda_u_history_Adam_Pretrain) + np.save( + f'Loss-Coe/{second_path}/lambda_v_history_Adam_Pretrain', + lambda_v_history_Adam_Pretrain) + np.save( + f'Loss-Coe/{second_path}/lambda_uxx_history_Adam_Pretrain', + lambda_uxx_history_Adam_Pretrain) + np.save( + f'Loss-Coe/{second_path}/lambda_uyy_history_Adam_Pretrain', + lambda_uyy_history_Adam_Pretrain) + np.save( + f'Loss-Coe/{second_path}/lambda_vxx_history_Adam_Pretrain', + lambda_vxx_history_Adam_Pretrain) + np.save( + f'Loss-Coe/{second_path}/lambda_vyy_history_Adam_Pretrain', + lambda_vyy_history_Adam_Pretrain) + + +if __name__ == "__main__": + parser = argparse.ArgumentParser() + parser.add_argument("--config_file_path", type=str, default="./config/ICNet_CKG.yaml") + args = parser.parse_known_args()[0] + + with open(args.config_file_path, 'r') as f: + config = yaml.safe_load(f) + + params = config.get('params', {}) + for key, value in params.items(): + setattr(args, key, value) + + model_name = args.model_name + case = args.case + device = args.device + device_id = args.device_id + network_size = args.network_size + learning_rate = args.learning_rate + epochs = args.epochs + BatchNo = args.BatchNo + load_params = args.load_params + second_path = args.second_path + description_CKG_u = args.description_CKG_u + description_CKG_v = args.description_CKG_v + lam = args.lam + d_tol = args.d_tol + + context.set_context( + mode=context.GRAPH_MODE, + save_graphs=False, + device_target=device, + device_id=device_id) + + use_npu = (device == 'Ascend') + + if use_npu: + msfloat_type = ms.float16 + else: + msfloat_type = ms.float32 + + X_u_train, u_train, v_train, X_f_train = read_training_data(args) + + model_pretrain = InvarianceConstrainedNN( + X_u_train, + u_train, + v_train, + X_f_train, + network_size, + BatchNo, + use_npu, + msfloat_type) + for epoch_train, lr_train in zip(epochs, learning_rate): + train(model_pretrain, int(float(epoch_train)), lr_train) + ms.save_checkpoint(model_pretrain.dnn, f'model/model_{lr}.ckpt') + + lambda_uxx_value = model_pretrain.lambda_uxx.numpy() + lambda_uyy_value = model_pretrain.lambda_uyy.numpy() + lambda_vxx_value = model_pretrain.lambda_vxx.numpy() + lambda_vyy_value = model_pretrain.lambda_vyy.numpy() + lambda_u_value = model_pretrain.lambda_u.numpy() + lambda_v_value = model_pretrain.lambda_v.numpy() + np.save(f'Loss-Coe/{second_path}/lambda_uxx_value', lambda_uxx_value) + np.save(f'Loss-Coe/{second_path}/lambda_uyy_value', lambda_uyy_value) + np.save(f'Loss-Coe/{second_path}/lambda_vxx_value', lambda_vxx_value) + np.save(f'Loss-Coe/{second_path}/lambda_vyy_value', lambda_vyy_value) + np.save(f'Loss-Coe/{second_path}/lambda_u_value', lambda_u_value) + np.save(f'Loss-Coe/{second_path}/lambda_v_value', lambda_v_value) + + if load_params: + lambda_u_value = np.load(f'Loss-Coe/{second_path}/lambda_u_value.npy') + lambda_v_value = np.load(f'Loss-Coe/{second_path}/lambda_v_value.npy') + lambda_uxx_value = np.load(f'Loss-Coe/{second_path}/lambda_uxx_value.npy') + lambda_uyy_value = np.load(f'Loss-Coe/{second_path}/lambda_uyy_value.npy') + lambda_vxx_value = np.load(f'Loss-Coe/{second_path}/lambda_vxx_value.npy') + lambda_vyy_value = np.load(f'Loss-Coe/{second_path}/lambda_vyy_value.npy') + model_ICCO = InvarianceConstrainedNN_STRdige(X_u_train, u_train, v_train, X_f_train, network_size, BatchNo, + lambda_u_value, lambda_v_value, lambda_uxx_value, lambda_uyy_value, lambda_vxx_value, lambda_vyy_value, + load_params, second_path, msfloat_type) + else: + model_ICCO = InvarianceConstrainedNN_STRdige(X_u_train, u_train, v_train, X_f_train, network_size, BatchNo, + lambda_u_value, lambda_v_value, lambda_uxx_value, lambda_uyy_value, lambda_vxx_value, lambda_vyy_value, + load_params, second_path, msfloat_type) + lam = 10**-5 + d_tol = 1 + lambda_u_STRidge, lambda_v_STRidge = model_ICCO.call_trainstridge(lam, d_tol) + + print_pde( + lambda_uxx_value, + lambda_uyy_value, + lambda_u_STRidge, + description_CKG_u, + ut='u_t') + + print_pde( + lambda_vxx_value, + lambda_vyy_value, + lambda_v_STRidge, + description_CKG_v, + ut='v_t') diff --git a/MindFlow/applications/research/ICNet/train_LI_2D_SKG.py b/MindFlow/applications/research/ICNet/train_LI_2D_SKG.py new file mode 100644 index 0000000000000000000000000000000000000000..a7bfe61109d0332114b26f798b225e4b7f86da2b --- /dev/null +++ b/MindFlow/applications/research/ICNet/train_LI_2D_SKG.py @@ -0,0 +1,193 @@ +# Copyright 2023 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================ +# pylint: disable=C0103 +"""ICNet train""" +import time +import argparse +import yaml +import numpy as np + +import mindspore as ms +from mindspore import set_seed, context, nn +from ChenChao.MindSpore.ICNet.ICNet.ICNet_PR.PR_003.src.network_skg import InvarianceConstrainedNN, InvarianceConstrainedNN_STRdige +from ChenChao.MindSpore.ICNet.ICNet.ICNet_PR.PR_003.src.datasets_skg import read_training_data, print_pde + + +# Set seed +np.random.seed(123456) +set_seed(123456) + + +def train(model, NIter, lr): + """train""" + # Get the gradients function + params_train = model.dnn.trainable_params() + params_train.append(model.lambda_u) + params_train.append(model.lambda_uxx) + params_train.append(model.lambda_uyy) + + optimizer_Adam = nn.Adam(params_train, learning_rate=lr) + + grad_fn = ms.value_and_grad(model.loss_fn, None, optimizer_Adam.parameters, has_aux=True) + + model.dnn.set_train() + + start_time = time.time() + + for epoch in range(1, 1 + NIter): + (loss, loss_u, loss_f_u, loss_lambda_u), grads = grad_fn( + model.x, model.y, model.t, model.x_f, model.y_f, model.t_f, model.u) + optimizer_Adam(grads) + + if epoch % 10 == 0: + elapsed = time.time() - start_time + print('It: %d, Loss: %.3e, lambda_u1: %.3f, lambda_u2: %.3f, lambda_u3: %.3f, lambda_uxx: %.3f, lambda_uyy: %.3f, Time: %.2f' % + (epoch, loss.item(), model.lambda_u[1].item(), model.lambda_u[2].item(), model.lambda_u[3].item(), model.lambda_uxx.item(), model.lambda_uyy.item(), elapsed)) + + initial_size = 4 + loss_history_Adam_Pretrain = np.empty([0]) + loss_u_history_Adam_Pretrain = np.empty([0]) + loss_f_u_history_Adam_Pretrain = np.empty([0]) + loss_lambda_u_history_Adam_Pretrain = np.empty([0]) + lambda_u_history_Adam_Pretrain = np.zeros((initial_size, 1)) + lambda_uxx_history_Adam_Pretrain = np.zeros((1, 1)) + lambda_uyy_history_Adam_Pretrain = np.zeros((1, 1)) + + loss_history_Adam_Pretrain = np.append(loss_history_Adam_Pretrain, loss.numpy()) + lambda_u_history_Adam_Pretrain = np.append( + lambda_u_history_Adam_Pretrain, model.lambda_u.numpy(), axis=1) + loss_u_history_Adam_Pretrain = np.append( + loss_u_history_Adam_Pretrain, loss_u.numpy()) + loss_f_u_history_Adam_Pretrain = np.append( + loss_f_u_history_Adam_Pretrain, loss_f_u.numpy()) + loss_lambda_u_history_Adam_Pretrain = np.append( + loss_lambda_u_history_Adam_Pretrain, loss_lambda_u.numpy()) + + lambda_uxx_new = np.array([model.lambda_uxx.numpy()]) + lambda_uyy_new = np.array([model.lambda_uyy.numpy()]) + lambda_uxx_history_Adam_Pretrain = np.append( + lambda_uxx_history_Adam_Pretrain, lambda_uxx_new, axis=1) + lambda_uyy_history_Adam_Pretrain = np.append( + lambda_uyy_history_Adam_Pretrain, lambda_uyy_new, axis=1) + + start_time = time.time() + + np.save(f'Loss-Coe/{second_path}/loss_history_Adam_Pretrain', loss_history_Adam_Pretrain) + np.save( + f'Loss-Coe/{second_path}/loss_u_history_Adam_Pretrain', + loss_u_history_Adam_Pretrain) + np.save( + f'Loss-Coe/{second_path}/loss_f_u_history_Adam_Pretrain', + loss_f_u_history_Adam_Pretrain) + np.save( + f'Loss-Coe/{second_path}/loss_lambda_u_history_Adam_Pretrain', + loss_lambda_u_history_Adam_Pretrain) + + np.save( + f'Loss-Coe/{second_path}/lambda_u_history_Adam_Pretrain', + lambda_u_history_Adam_Pretrain) + np.save( + f'Loss-Coe/{second_path}/lambda_uxx_history_Adam_Pretrain', + lambda_uxx_history_Adam_Pretrain) + np.save( + f'Loss-Coe/{second_path}/lambda_uyy_history_Adam_Pretrain', + lambda_uyy_history_Adam_Pretrain) + + +if __name__ == "__main__": + parser = argparse.ArgumentParser() + parser.add_argument("--config_file_path", type=str, default="./config/ICNet_SKG.yaml") + args = parser.parse_known_args()[0] + + with open(args.config_file_path, 'r') as f: + config = yaml.safe_load(f) + + params = config.get('params', {}) + for key, value in params.items(): + setattr(args, key, value) + + model_name = args.model_name + case = args.case + device = args.device + device_id = args.device_id + network_size = args.network_size + learning_rate = args.learning_rate + epochs = args.epochs + BatchNo = args.BatchNo + load_params = args.load_params + second_path = args.second_path + description_SKG = args.description_SKG + lam = args.lam + d_tol = args.d_tol + + context.set_context( + mode=context.GRAPH_MODE, + save_graphs=False, + device_target=device, + device_id=device_id) + + use_npu = (device == 'Ascend') + + if use_npu: + msfloat_type = ms.float16 + else: + msfloat_type = ms.float32 + + X_u_train, u_train, X_f_train = read_training_data(args) + + model_pretrain = InvarianceConstrainedNN( + X_u_train, + u_train, + X_f_train, + network_size, + BatchNo, + use_npu, + msfloat_type) + for epoch_train, lr_train in zip(epochs, learning_rate): + train(model_pretrain, int(float(epoch_train)), lr_train) + ms.save_checkpoint(model_pretrain.dnn, f'model/model.ckpt') + + lambda_uxx_value = model_pretrain.lambda_uxx.numpy() + lambda_uyy_value = model_pretrain.lambda_uyy.numpy() + lambda_u_value = model_pretrain.lambda_u.numpy() + + np.save(f'Loss-Coe/{Second_path}/lambda_uxx_value', lambda_uxx_value) + np.save(f'Loss-Coe/{Second_path}/lambda_uyy_value', lambda_uyy_value) + np.save(f'Loss-Coe/{Second_path}/lambda_u_value', lambda_u_value) + + if load_params: + lambda_u_value = np.load(f'Loss-Coe/{second_path}/lambda_u_value.npy') + lambda_uxx_value = np.load(f'Loss-Coe/{second_path}/lambda_uxx_value.npy') + lambda_uyy_value = np.load(f'Loss-Coe/{second_path}/lambda_uyy_value.npy') + model_ICCO = InvarianceConstrainedNN_STRdige( + X_u_train, + u_train, + X_f_train, + network_size, + BatchNo, + lambda_u_value, + lambda_uxx_value, + lambda_uyy_value, + load_params, + second_path, + msfloat_type) + else: + model_ICCO = InvarianceConstrainedNN_STRdige(X_u_train, u_train, X_f_train, network_size, BatchNo, + lambda_u_value, lambda_uxx_value, lambda_uyy_value, load_params, second_path, msfloat_type) + lam = 10**-5 + d_tol = 5 + lambda_u_STRidge = model_ICCO.call_trainstridge(lam, d_tol) + + print_pde(lambda_uxx_value, lambda_uyy_value, lambda_u_STRidge, description_ks, ut='u_t')