diff --git a/product/en/docs-mogdb/v3.1/administrator-guide/localization/character-set-support.md b/product/en/docs-mogdb/v3.1/administrator-guide/localization/character-set-support.md index 43265a8f8e989b1cd4b09f54405160b2f220cb07..f6245b34f724c5f7092ae4eb5370aceb25726d96 100644 --- a/product/en/docs-mogdb/v3.1/administrator-guide/localization/character-set-support.md +++ b/product/en/docs-mogdb/v3.1/administrator-guide/localization/character-set-support.md @@ -106,46 +106,47 @@ MogDB supports automatic character set conversion between server and client for | Server Character Set | Available Client Character Sets | | -------------------- | ------------------------------------------------------------ | -| `BIG5` | not supported as a server encoding | -| `EUC_CN` | EUC_CN, `MULE_INTERNAL`, `UTF8` | -| `EUC_JP` | EUC_JP, `MULE_INTERNAL`, `SJIS`, `UTF8` | -| `EUC_JIS_2004` | EUC_JIS_2004, `SHIFT_JIS_2004`, `UTF8` | -| `EUC_KR` | EUC_KR, `MULE_INTERNAL`, `UTF8` | -| `EUC_TW` | EUC_TW, `BIG5`, `MULE_INTERNAL`, `UTF8` | -| `GB18030` | not supported as a server encoding | -| `GBK` | not supported as a server encoding | -| `ISO_8859_5` | ISO_8859_5, `KOI8R`, `MULE_INTERNAL`, `UTF8`, `WIN866`, `WIN1251` | -| `ISO_8859_6` | ISO_8859_6, `UTF8` | -| `ISO_8859_7` | ISO_8859_7, `UTF8` | -| `ISO_8859_8` | ISO_8859_8, `UTF8` | -| `JOHAB` | not supported as a server encoding | -| `KOI8R` | KOI8R, `ISO_8859_5`, `MULE_INTERNAL`, `UTF8`, `WIN866`, `WIN1251` | -| `KOI8U` | KOI8U, `UTF8` | -| `LATIN1` | LATIN1, `MULE_INTERNAL`, `UTF8` | -| `LATIN2` | LATIN2, `MULE_INTERNAL`, `UTF8`, `WIN1250` | -| `LATIN3` | LATIN3, `MULE_INTERNAL`, `UTF8` | -| `LATIN4` | LATIN4, `MULE_INTERNAL`, `UTF8` | -| `LATIN5` | LATIN5, `UTF8` | -| `LATIN6` | LATIN6, `UTF8` | -| `LATIN7` | LATIN7, `UTF8` | -| `LATIN8` | LATIN8, `UTF8` | -| `LATIN9` | LATIN9, `UTF8` | -| `LATIN10` | LATIN10, `UTF8` | -| `MULE_INTERNAL` | MULE_INTERNAL, `BIG5`, `EUC_CN`, `EUC_JP`, `EUC_KR`, `EUC_TW`, `ISO_8859_5`, `KOI8R`, `LATIN1` to `LATIN4`, `SJIS`, `WIN866`, `WIN1250`, `WIN1251` | -| `SJIS` | not supported as a server encoding | -| `SHIFT_JIS_2004` | not supported as a server encoding | -| `SQL_ASCII` | any (no conversion will be performed) | -| `UHC` | not supported as a server encoding | +| `BIG5` | `BIG5`,`EUC_TW`,`MULE_INTERNAL`,`UTF8` | +| `EUC_CN` | `EUC_CN`,`MULE_INTERNAL`,`UTF8` | +| `EUC_JIS_2004` | `EUC_JIS_2004`,`UTF8`,`SHIFT_JIS_2004` | +| `EUC_JP` | `EUC_JP`, `MULE_INTERNAL`, `SJIS`, `UTF8` | +| `EUC_KR` | `EUC_KR`, `MULE_INTERNAL`, `UTF8` | +| `EUC_TW` | `EUC_TW`, `BIG5`, `MULE_INTERNAL`, `UTF8` | +| `GB18030` | `GB18030`,`UTF8` | +| `GBK` | `GBK`,`UTF8` | +| `ISO_8859_5` | `ISO_8859_5`, `KOI8R`, `MULE_INTERNAL`, `UTF8`, `WIN866`, `WIN1251` | +| `ISO_8859_6` | `ISO_8859_6`, `UTF8` | +| `ISO_8859_7` | `ISO_8859_7`, `UTF8` | +| `ISO_8859_8` | `ISO_8859_8`, `UTF8` | +| `JOHAB` | `JOHAB`, `UTF8` | +| `KOI8R` | `KOI8R`, `ISO_8859_5`, `MULE_INTERNAL`, `UTF8`, `WIN866`, `WIN1251` | +| `KOI8U` | `KOI8U`, `UTF8` | +| `LATIN1` | `LATIN1`, `MULE_INTERNAL`, `UTF8` | +| `LATIN2` | `LATIN2`, `MULE_INTERNAL`, `UTF8`, `WIN1250` | +| `LATIN3` | `LATIN3`, `MULE_INTERNAL`, `UTF8` | +| `LATIN4` | `LATIN4`, `MULE_INTERNAL`, `UTF8` | +| `LATIN5` | `LATIN5`, `UTF8` | +| `LATIN6` | `LATIN6`, `UTF8` | +| `LATIN7` | `LATIN7`, `UTF8` | +| `LATIN8` | `LATIN8`, `UTF8` | +| `LATIN9` | `LATIN9`, `UTF8` | +| `LATIN10` | `LATIN10`, `UTF8` | +| `MULE_INTERNAL` | `MULE_INTERNAL`,`SQL_ASCII`,`KOI8R`,`ISO_8859_5`,`WIN1251`,`WIN866`,`EUC_CN`,`EUC_JP`,`SJIS`,`EUC_KR`,`EUC_TW`,`BIG5`,`LATIN2`,`WIN1250`,`LATIN1`,`LATIN3`,`LATIN4` | +| `SJIS` | `SJIS`,`EUC_JP`,`MULE_INTERNAL`,`UTF8` | +| `SQL_ASCII` | `SQL_ASCII`,`MULE_INTERNAL`,`UTF8` | +| `UHC` | `UHC`,`UTF8` | | `UTF8` | all supported encodings | -| `WIN866` | WIN866, `ISO_8859_5`, `KOI8R`, `MULE_INTERNAL`, `UTF8`, `WIN1251` | -| `WIN874` | WIN874, `UTF8` | -| `WIN1250` | WIN1250, `LATIN2`, `MULE_INTERNAL`, `UTF8` | -| `WIN1251` | WIN1251, `ISO_8859_5`, `KOI8R`, `MULE_INTERNAL`, `UTF8`, `WIN866` | -| `WIN1252` | WIN1252, `UTF8` | -| `WIN1253` | WIN1253, `UTF8` | -| `WIN1254` | WIN1254, `UTF8` | -| `WIN1255` | WIN1255, `UTF8` | -| `WIN1256` | WIN1256, `UTF8` | +| `WIN866` | `WIN866`, `ISO_8859_5`, `KOI8R`, `MULE_INTERNAL`, `UTF8`, `WIN1251` | +| `WIN874` | `WIN874`, `UTF8` | +| `WIN1250` | `WIN1250`, `LATIN2`, `MULE_INTERNAL`, `UTF8` | +| `WIN1251` | `WIN1251`, `ISO_8859_5`, `KOI8R`, `MULE_INTERNAL`, `UTF8`, `WIN866` | +| `WIN1252` | `WIN1252`, `UTF8` | +| `WIN1253` | `WIN1253`, `UTF8` | +| `WIN1254` | `WIN1254`, `UTF8` | +| `WIN1255` | `WIN1255`, `UTF8` | +| `WIN1256` | `WIN1256`, `UTF8` | +| `WIN1257` | `WIN1257`, `UTF8` | +| `WIN1258` | `WIN1258`, `UTF8` | To enable automatic character set conversion, you have to tell MogDB the character set (encoding) you would like to use in the client. There are several ways to accomplish this: diff --git a/product/en/docs-mogdb/v5.0/AI-features/1-AI-features-overview.md b/product/en/docs-mogdb/v5.0/AI-features/ai-feature.md similarity index 87% rename from product/en/docs-mogdb/v5.0/AI-features/1-AI-features-overview.md rename to product/en/docs-mogdb/v5.0/AI-features/ai-feature.md index 130d3caa1b854027e76029116c47426840e52d35..a59d962b907f6a936fe288f8d8b42a368ee290aa 100644 --- a/product/en/docs-mogdb/v5.0/AI-features/1-AI-features-overview.md +++ b/product/en/docs-mogdb/v5.0/AI-features/ai-feature.md @@ -9,10 +9,12 @@ date: 2021-05-19 The history of artificial intelligence (AI) can be dated back to as early as the 1950s, even longer than the history of the database development. However, the AI technology has not been applied on a large scale for a long time due to various objective factors, and even experienced several obvious troughs. With the further development of information technologies in recent years, factors that restrict the AI development have been gradually weakened, and the AI, big data, and cloud computing (ABC) technologies are born. -The combination of AI and databases has been a trending research topic in the industry in recent years. MogDB has participated in the exploration of this domain earlier and achieved phased achievements. An AI submodule DBMind is provided for the database. Compared with other functions, it is more independent. This module can be divided into AI4DB, DB4AI, and AI in DB. +The combination of AI and databases has been a trending research topic in the industry in recent years. MogDB has participated in the exploration of this domain earlier and achieved phased achievements. An AI submodule DBMind is provided for the database. Compared with other functions, it is more independent. This module can be divided into AI4DB and DB4AI. - AI4DB uses AI technologies to optimize database execution performance as well as achieve autonomy and O&M free. It includes self-tuning, self-diagnosis, self-security, self-O&M, and self-healing. - DB4AI streamlines the E2E process from databases to AI applications, drives AI tasks through databases, and unifies the AI technology stack to achieve out-of-the-box, high performance, and cost saving. For example, SQL-like statements are used to implement functions such as recommendation system, image retrieval, and time series prediction. The advantages of high parallelism and column store of databases can be fully utilized to avoid the cost of data and fragmented storage and avoid security risks caused by information leakage. -- AI in DB modifies the database kernel to implement functions that cannot be implemented in the original database architecture. For example, AI algorithms are used to improve the database optimizer to implement more accurate cost estimation. The functions described in this section are stored in the **bin/dbmind** directory of the database installation directory (*$GAUSSHOME*). The sub-functions are stored in the **components** subdirectory of **bin/dbmind**. To invoke DBMind, you can run the **gs_dbmind** command. In addition, the built-in AI functions (such as DB4AI) of the database are presented in the form of SQL syntax and system functions. + +- **[AI4DB Autonomous Database O&M](./ai4db/ai4db-autonomous-database-o&m.md)** +- **[DB4AI Database-driven AI](./db4ai/db4ai.md)** \ No newline at end of file diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai-in-db/intelligence-explain/intelligence-explain-best-practices.md b/product/en/docs-mogdb/v5.0/AI-features/ai-in-db/intelligence-explain/intelligence-explain-best-practices.md deleted file mode 100644 index 966e32a56ea86d6f0656c4b4a303d4da1af3e049..0000000000000000000000000000000000000000 --- a/product/en/docs-mogdb/v5.0/AI-features/ai-in-db/intelligence-explain/intelligence-explain-best-practices.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -title: Best Practices -summary: Best Practices -author: Guo Huan -date: 2022-05-06 ---- - -# Best Practices - -For details about the parameters, see [GS_OPT_MODEL](../../../reference-guide/system-catalogs-and-system-views/system-catalogs/GS_OPT_MODEL.md)。 - -**Table 1** - -| Model Parameter | Recommended Value | -| :--------------- | :----------------------------------------------------------- | -| template_name | ‘rlstm’ | -| model_name | The value can be customized, for example, **open_ai**. The value must meet the unique constraint. | -| datname | Name of the database to be served, for example, **postgres**. | -| ip | IP address of the AI Engine, for example, **127.0.0.1**. | -| port | AI Engine listening port number, for example, **5000**. | -| max_epoch | Iteration times. A large value is recommended to ensure the convergence effect, for example, **2000**. | -| learning_rate | (0, 1] is a floating-point number. A large learning rate is recommended to accelerate convergence. | -| dim_red | Number of feature values to be reduced. **-1**: Do not use PCA for dimension reduction. All features are supported. Floating point number in the range of (0, 1]: A smaller value indicates a smaller training dimension and a faster convergence speed, but affects the training accuracy. | -| hidden_units | If the feature value dimension is high, you are advised to increase the value of this parameter to increase the model complexity. For example, set this parameter to **64**, **128**, and so on. | -| batch_size | You are advised to increase the value of this parameter based on the amount of encoded data to accelerate model convergence. For example, set this parameter to **256**, **512**, and so on. | -| Other parameters | See [GS_OPT_MODEL](../../../reference-guide/system-catalogs-and-system-views/system-catalogs/GS_OPT_MODEL.md) | - -Recommended parameter settings: - -``` -INSERT INTO gs_opt_model values('rlstm', 'open_ai', 'postgres', '127.0.0.1', 5000, 2000，1, -1, 64, 512, 0 , false, false, '{S, T}', '{0,0}', '{0,0}', 'Text'); -``` \ No newline at end of file diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai-in-db/intelligence-explain/intelligence-explain-environment-deployment.md b/product/en/docs-mogdb/v5.0/AI-features/ai-in-db/intelligence-explain/intelligence-explain-environment-deployment.md deleted file mode 100644 index ee45fd939a9ae54359a26cb462938e3f7f1e78b4..0000000000000000000000000000000000000000 --- a/product/en/docs-mogdb/v5.0/AI-features/ai-in-db/intelligence-explain/intelligence-explain-environment-deployment.md +++ /dev/null @@ -1,203 +0,0 @@ ---- -title: Environment Deployment -summary: Environment Deployment -author: Guo Huan -date: 2022-05-06 ---- - -# Environment Deployment - -## Prerequisites - -MogDB is in the normal state. The user has logged in to MogDB with an authenticated identity. The executed SQL syntax is correct and no error is reported. In the historical performance data window, the number of MogDB concurrent tasks is stable, the structure and number of tables remain unchanged, the data volume changes smoothly, and the GUC parameters related to query performance remain unchanged. During prediction, the model has been trained and converged. The running environment of the AIEngine is stable. - -## Request Example - -The AIEngine process communicates with the kernel process using HTTPS. An example request is as follows: - -```bash -curl -X POST -d '{"modelName":"modelname"}' -H 'Content-Type: application/json' https://IP-address:port/request-API -``` - -**Table 1** AIEngine external APIs - -| Request API | Description | -| :------------- | :------------------------------------------ | -| /check | Checks whether a model is properly started. | -| /configure | Sets model parameters. | -| /train | Trains a model. | -| /track_process | Views model training logs. | -| /setup | Loads historical models. | -| /predict | Predicts a model. | - -## Generating Certificates - -Before using the prediction function, you need to use OpenSSL to generate certificates required for authentication between the communication parties, ensuring communication security. - -1. Set up a certificate generation environment. The certificate file is stored in **$GAUSSHOME/CA**. - - Copy the certificate generation script and related files. - - ```bash - cp path_to_predictor/install/ssl.sh $GAUSSHOME/ - cp path_to_predictor/install/ca_ext.txt $GAUSSHOME/ - ``` - - Copy the configuration file **openssl.cnf** to **$GAUSSHOME**. - - ```bash - cp $GAUSSHOME/share/om/openssl.cnf $GAUSSHOME/ - ``` - - Modify the configuration parameters in **openssl.conf**. - - ```bash - dir = $GAUSSHOME/CA/demoCA - default_md = sha256 - ``` - - The certificate generation environment is ready. - -2. Generate a certificate and private key. - - ```bash - cd $GAUSSHOME - sh ssl.sh - ``` - - Set the password as prompted, for example, **Test@123**. - - The password must contain at least eight characters of at least three different types. - - ```bash - Please enter your password: - ``` - - Set the options as prompted. - - ```bash - Certificate Details: - Serial Number: 1 (0x1) - Validity - Not Before: May 15 08:32:44 2020 GMT - Not After : May 15 08:32:44 2021 GMT - Subject: - countryName = CN - stateOrProvinceName = SZ - organizationName = HW - organizationalUnitName = GS - commonName = CA - X509v3 extensions: - X509v3 Basic Constraints: - CA:TRUE - Certificate is to be certified until May 15 08:32:44 2021 GMT (365 days) - Sign the certificate? [y/n]:y - 1 out of 1 certificate requests certified, commit? [y/n]y - ``` - - Enter the IP address for starting the AIEngine, for example, **127.0.0.1**. - - ```bash - Please enter your aiEngine IP: 127.0.0.1 - ``` - - Set the options as prompted. - - ```bash - Certificate Details: - Serial Number: 2 (0x2) - Validity - Not Before: May 15 08:38:07 2020 GMT - Not After : May 13 08:38:07 2030 GMT - Subject: - countryName = CN - stateOrProvinceName = SZ - organizationName = HW - organizationalUnitName = GS - commonName = 127.0.0.1 - X509v3 extensions: - X509v3 Basic Constraints: - CA:FALSE - Certificate is to be certified until May 13 08:38:07 2030 GMT (3650 days) - Sign the certificate? [y/n]:y - 1 out of 1 certificate requests certified, commit? [y/n]y - ``` - - Enter the IP address for starting MogDB, for example, **127.0.0.1**. - - ```bash - Please enter your mogdb IP: 127.0.0.1 - ``` - - Set the options as prompted. - - ```bash - Certificate Details: - Serial Number: 3 (0x3) - Validity - Not Before: May 15 08:41:46 2020 GMT - Not After : May 13 08:41:46 2030 GMT - Subject: - countryName = CN - stateOrProvinceName = SZ - organizationName = HW - organizationalUnitName = GS - commonName = 127.0.0.1 - X509v3 extensions: - X509v3 Basic Constraints: - CA:FALSE - Certificate is to be certified until May 13 08:41:46 2030 GMT (3650 days) - Sign the certificate? [y/n]:y - 1 out of 1 certificate requests certified, commit? [y/n]y - ``` - - The related certificate and key have been generated. The content in **$GAUSSHOME/CA** is as follows: - - ![img](https://cdn-mogdb.enmotech.com/docs-media/mogdb/developer-guide/intelligence-explain-1.png) - -## Setting Up the Environment - -1. Copy the tool code folder to the target environment. - - – Assume that the installation directory is **$INSTALL_FOLDER**. - - – Assume that the destination directory is **/home/ai_user**. - - ```bash - scp -r $INSTALL_FOLDER/bin/dbmind/predictor ai_user@127.0.0.1:path_to_Predictor - ``` - -2. Copy the CA certificate folder to a directory in the AIEngine environment. - - ```bash - cp -r $GAUSSHOME/CA ai_user@127.0.0.1:path_to_CA - ``` - -3. Install the **predictor/install/requirements(-gpu).txt** tool. - - ```bash - With GPU: pip install -r requirements-gpu.txt - Without GPU: pip install -r requirements.txt - ``` - -## Starting the AIEngine - -1. Switch to the AIEngine environment (that is, copy the target environment **ai_user** of the predictor). - - Set parameters in **predictor/python/settings.py**. - - ```bash - DEFAULT_FLASK_SERVER_HOST = '127.0.0.1' (running IP address of the AIEngine) - DEFAULT_FLASK_SERVER_PORT = '5000' (running port number of the AIEngine) - PATH_SSL = "path_to_CA" (CA folder path) - ``` - -2. Run the startup script of the AIEngine. - - ```bash - python path_to_Predictor/python/run.py - ``` - - In this case, the AIEngine keeps enabled on the corresponding port and waits for the request of the time prediction function from the kernel. - - For details about how to initiate a time prediction command from the kernel, see[User Guide](intelligence-explain-user-guide.md)。 diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai-in-db/intelligence-explain/intelligence-explain-faqs.md b/product/en/docs-mogdb/v5.0/AI-features/ai-in-db/intelligence-explain/intelligence-explain-faqs.md deleted file mode 100644 index ef6e404d02fecdda73e32f6cf733e4507f2f6c30..0000000000000000000000000000000000000000 --- a/product/en/docs-mogdb/v5.0/AI-features/ai-in-db/intelligence-explain/intelligence-explain-faqs.md +++ /dev/null @@ -1,36 +0,0 @@ ---- -title: FAQs -summary: FAQs -author: Guo Huan -date: 2022-05-06 ---- - -# FAQs - -## AI Engine Configuration - -- **AI Engine failed to be started**: Check whether the IP address and port are available and whether the CA certificate path exists. -- **AI Engine does not respond**: Check whether the CA certificates of the two communication parties are consistent. -- **Training and test failure**: Check whether the path for saving the model files exists and whether the training prediction file is correctly downloaded. - -- **Changing the AI Engine IP address**: Regenerate the certificate by following the steps in [Generating Certificates](intelligence-explain-environment-deployment.md#证书生成). Enter the new IP address in Generate a certificate and private key. - -## Database Internal Errors - -Problem: AI Engine connection failed. - -``` -ERROR: AI engine connection failed. -CONTEXT: referenced column: model_train_opt -``` - -Solution: Check whether the AI Engine is started or restarted properly. Check whether the CA certificates of the communication parties are consistent. Check whether the IP address and port number in the model configuration match. - -Problem: The model does not exist. - -``` -ERROR: OPT_Model not found for model name XXX -CONTEXT: referenced column: track_model_train_opt -``` - -Solution: Check whether [GS_OPT_MODEL](../../../reference-guide/system-catalogs-and-system-views/system-catalogs/GS_OPT_MODEL.md) contains the model specified in the **model_name** column in the statement. If the error is reported when the prediction function is used, check whether the model has been trained. diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai-in-db/intelligence-explain/intelligence-explain-overview.md b/product/en/docs-mogdb/v5.0/AI-features/ai-in-db/intelligence-explain/intelligence-explain-overview.md deleted file mode 100644 index e61176a69f05f203394bbca616584e7db9ea1370..0000000000000000000000000000000000000000 --- a/product/en/docs-mogdb/v5.0/AI-features/ai-in-db/intelligence-explain/intelligence-explain-overview.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -title: Overview -summary: Overview -author: Guo Huan -date: 2022-05-06 ---- - -# Overview - -Predictor is a query time prediction tool that leverages machine learning and has online learning capabilities. By continuously learning the historical execution information collected in the database, Predictor can predict the execution time of a plan. - -To use this tool, you must start the Python process AI Engine for model training and inference. - -This function is stored in the *$GAUSSHOME***/bin/dbmind/components/predictor** directory. Some functions of this module involve complex setup. Therefore, you need to switch to the directory to find the corresponding files and deploy the functions according to the description in this section. diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai-in-db/intelligence-explain/intelligence-explain-user-guide.md b/product/en/docs-mogdb/v5.0/AI-features/ai-in-db/intelligence-explain/intelligence-explain-user-guide.md deleted file mode 100644 index 062231cae19eb920967fe885b4a186c648175254..0000000000000000000000000000000000000000 --- a/product/en/docs-mogdb/v5.0/AI-features/ai-in-db/intelligence-explain/intelligence-explain-user-guide.md +++ /dev/null @@ -1,205 +0,0 @@ ---- -title: User Guide -summary: User Guide -author: Guo Huan -date: 2022-05-06 ---- - -# User Guide - -## Data Collection - -1. Enable data collection. - - 1. Set parameters related to the ActiveSQL operator. - - ``` - enable_resource_track=on - resource_track_level=operator - enable_resource_record=on - resource_track_cost=10 (The default value is 100000.) - ``` - - > ![img](https://cdn-mogdb.enmotech.com/docs-media/icon/icon-note.gif) **NOTE:** - > - > - The value of **resource_track_cost** must be smaller than the total query cost of the information to be collected. Only the information that meets the requirements can be collected. - > - Cgroup functions are available. - - 2. Collect information. - - Execute the service query statement. - - View data collected in real time. - - ``` - select * from gs_wlm_plan_operator_history; - ``` - - Expected result: All jobs that meet **resource_track_duration** and **resource_track_cost** are collected. - -2. Disable data collection. - - 1. Set parameters related to the ActiveSQL operator. - - ``` - enable_resource_track=off - resource_track_level=none - resource_track_level=query - ``` - - 2. Execute the service query statement. - - Wait for 3 minutes and check the data on the current node. - - ``` - select * from gs_wlm_plan_operator_info; - ``` - - Expected result: No new data is added to the tables and views. - -3. Persist data. - - 1. Set parameters related to the ActiveSQL operator. - - ``` - enable_resource_track=on - resource_track_level=operator - enable_resource_record=on - resource_track_duration=0 (The default value is 60s.) - resource_track_cost=10 (The default value is 100000.) - ``` - - > ![img](https://cdn-mogdb.enmotech.com/docs-media/icon/icon-note.gif) **NOTE:** - > - > - The value of **resource_track_cost** must be smaller than the total query cost of the information to be collected. Only the information that meets the requirements can be collected. - > - Cgroup functions are available. - - 2. Execute the service query statement. - - Wait for 3 minutes and check the data on the current node. - - ``` - select * from gs_wlm_plan_operator_info; - ``` - - Expected result: All jobs that meet **resource_track_duration** and **resource_track_cost** are collected. - -## Model Management (System Administrators) - -> ![img](https://cdn-mogdb.enmotech.com/docs-media/icon/icon-note.gif) **NOTE:** Model management operations can be performed only when the database is normal. - -1. Add a new model. - - INSERT INTO gs_opt_model values('……')； - - Example: - - ``` - INSERT INTO gs_opt_model values('rlstm', 'model_name', 'datname', '127.0.0.1', 5000, 2000, 1, -1, 64, 512, 0 , false, false, '{S, T}', '{0,0}', '{0,0}', 'Text'); - ``` - - > ![img](https://cdn-mogdb.enmotech.com/docs-media/icon/icon-note.gif) **NOTE:** - > - > - For details about model parameter settings, see [GS_OPT_MODEL](../../../reference-guide/system-catalogs-and-system-views/system-catalogs/GS_OPT_MODEL.md). - > - Currently, only **rlstm** is supported in the **template_name** column. - > - The values in the **datname** column must be the same as those in the database used for model usage and training. Otherwise, the model cannot be used. - > - The values in the **model_name** column must meet the **unique** constraint. - > - For details about other parameter settings, see [Best Practices](intelligence-explain-best-practices.md). - -2. Modify model parameters. - - ``` - UPDATE gs_opt_model SET = WHERE model_name = ; - ``` - -3. Delete a model. - - ``` - DELETE FROM gs_opt_model WHERE model_name = ; - ``` - -4. Query the existing model and its status. - - ``` - SELECT * FROM gs_opt_model; - ``` - -## Model Training (System Administrators) - -1. Add models and modify model parameters by following the steps in Model Management (System Administrators). - - Example: - - Add a mode. - - ``` - INSERT INTO gs_opt_model values('rlstm', 'default', 'postgres', '127.0.0.1', 5000, 2000, 1, -1, 64, 512, 0 , false, false, '{S, T}', '{0,0}', '{0,0}', 'Text'); - ``` - - Modify training parameters. - - ``` - UPDATE gs_opt_model SET = WHERE model_name = ; - ``` - -2. Check that the database status is normal and historical data is collected properly before you perform the following operations: - - Delete the original encoding data. - - ``` - DELETE FROM gs_wlm_plan_encoding_table; - ``` - - To encode data, specify the database name. - - ``` - SELECT gather_encoding_info('postgres'); - ``` - - Start training. - - ``` - SELECT model_train_opt('rlstm', 'default'); - ``` - -3. Run the following command to obtain the relative path of the model training log on the AIEngine side: - - ``` - SELECT * FROM track_model_train_opt('rlstm', 'default'); - ``` - -## Model Prediction - -> ![img](https://cdn-mogdb.enmotech.com/docs-media/icon/icon-note.gif) **NOTE:** -> -> - Model prediction can be performed only when the database status is normal and the specified model has been trained and converged. -> -> - Currently, the labels of model training parameters must contain the **S** label so that the **p-time** value can be displayed in **EXPLAIN**. Example: INSERT INTO gs_opt_model values('rlstm', 'default', 'postgres', '127.0.0.1', 5000, 1000, 1, -1, 50, 500, 0 , false, false, '{**S**, T}', '{0,0}', '{0,0}', 'Text'); - -1. Call EXPLAIN. - - ``` - explain (analyze on, predictor ) - SELECT ... - ``` - - Expected result: - - ``` - Example: Row Adapter (cost=110481.35..110481.35 rows=100 p-time=99..182 width=100) (actual time=375.158..375.160 rows=2 loops=1) - The p-time column indicates the predicted value of the label. - ``` - -## Other Functions - -1. Check whether the AIEngine can be connected. - - ``` - MogDB=# select check_engine_status('aiEngine-ip-address',running-port); - ``` - -2. Check the path for storing model logs on the AIEngine. - - ``` - MogDB=# select track_model_train_opt('template_name', 'model_name'); - ``` diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/abo-optimizer/abo-optimizer.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/abo-optimizer/abo-optimizer.md new file mode 100644 index 0000000000000000000000000000000000000000..2758ee7ede25c95d62caf69dbaa601b4b483dba7 --- /dev/null +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/abo-optimizer/abo-optimizer.md @@ -0,0 +1,12 @@ +--- +title: ABO Optimizer +summary: ABO Optimizer +author: zhang cuiping +date: 2023-04-07 +--- + +# ABO Optimizer + +- **[intelligent-cardinality-estimation](./intelligent-cardinality-estimation/ai4db-intelligent-cardinality-estimation.md)** + +- **[adaptive-plan-selection](./adaptive-plan-selection/ai4db-adaptive-plan-selection.md)** \ No newline at end of file diff --git a/product/en/docs-mogdb/v5.0/AI-features/abo-optimizer/adaptive-plan-selection/adaptive-plan-selection-best-practices.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/abo-optimizer/adaptive-plan-selection/adaptive-plan-selection-best-practices.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/abo-optimizer/adaptive-plan-selection/adaptive-plan-selection-best-practices.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/abo-optimizer/adaptive-plan-selection/adaptive-plan-selection-best-practices.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/abo-optimizer/adaptive-plan-selection/adaptive-plan-selection-overview.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/abo-optimizer/adaptive-plan-selection/adaptive-plan-selection-overview.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/abo-optimizer/adaptive-plan-selection/adaptive-plan-selection-overview.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/abo-optimizer/adaptive-plan-selection/adaptive-plan-selection-overview.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/abo-optimizer/adaptive-plan-selection/adaptive-plan-selection-prerequisites.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/abo-optimizer/adaptive-plan-selection/adaptive-plan-selection-prerequisites.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/abo-optimizer/adaptive-plan-selection/adaptive-plan-selection-prerequisites.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/abo-optimizer/adaptive-plan-selection/adaptive-plan-selection-prerequisites.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/abo-optimizer/adaptive-plan-selection/adaptive-plan-selection-troubleshooting.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/abo-optimizer/adaptive-plan-selection/adaptive-plan-selection-troubleshooting.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/abo-optimizer/adaptive-plan-selection/adaptive-plan-selection-troubleshooting.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/abo-optimizer/adaptive-plan-selection/adaptive-plan-selection-troubleshooting.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/abo-optimizer/adaptive-plan-selection/adaptive-plan-selection-usage-guide.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/abo-optimizer/adaptive-plan-selection/adaptive-plan-selection-usage-guide.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/abo-optimizer/adaptive-plan-selection/adaptive-plan-selection-usage-guide.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/abo-optimizer/adaptive-plan-selection/adaptive-plan-selection-usage-guide.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/abo-optimizer/adaptive-plan-selection/ai4db-adaptive-plan-selection.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/abo-optimizer/adaptive-plan-selection/ai4db-adaptive-plan-selection.md new file mode 100644 index 0000000000000000000000000000000000000000..e472d25059824947dcc58fe20a858a437fccbb7e --- /dev/null +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/abo-optimizer/adaptive-plan-selection/ai4db-adaptive-plan-selection.md @@ -0,0 +1,18 @@ +--- +title: adaptive-plan-selection +summary: adaptive-plan-selection +author: zhang cuiping +date: 2023-04-07 +--- + +# Adaptive Plan Selection + +- **[Overview](adaptive-plan-selection-overview.md)** + +- **[Prerequisites](adaptive-plan-selection-prerequisites.md)** + +- **[Usage Guide](adaptive-plan-selection-usage-guide.md)** + +- **[Best Practice](adaptive-plan-selection-best-practices.md)** + +- **[Troubleshooting](adaptive-plan-selection-troubleshooting.md)** \ No newline at end of file diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/abo-optimizer/intelligent-cardinality-estimation/ai4db-intelligent-cardinality-estimation.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/abo-optimizer/intelligent-cardinality-estimation/ai4db-intelligent-cardinality-estimation.md new file mode 100644 index 0000000000000000000000000000000000000000..ab43288d79a4d28eb05b7a1ee2c3b7f694f619b7 --- /dev/null +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/abo-optimizer/intelligent-cardinality-estimation/ai4db-intelligent-cardinality-estimation.md @@ -0,0 +1,18 @@ +--- +title: intelligent-cardinality-estimation +summary: intelligent-cardinality-estimation +author: zhang cuiping +date: 2023-04-07 +--- + +# Intelligent Cardinality Estimation + +- **[Overview](intelligent-cardinality-estimation-overview.md)** + +- **[Prerequisites](intelligent-cardinality-estimation-prerequisites.md)** + +- **[Usage Guide](intelligent-cardinality-estimation-usage-guide.md)** + +- **[Best Practice](intelligent-cardinality-estimation-best-practices.md)** + +- **[Troubleshooting](intelligent-cardinality-estimation-troubleshooting.md)** \ No newline at end of file diff --git a/product/en/docs-mogdb/v5.0/AI-features/abo-optimizer/intelligent-cardinality-estimation/intelligent-cardinality-estimation-best-practices.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/abo-optimizer/intelligent-cardinality-estimation/intelligent-cardinality-estimation-best-practices.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/abo-optimizer/intelligent-cardinality-estimation/intelligent-cardinality-estimation-best-practices.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/abo-optimizer/intelligent-cardinality-estimation/intelligent-cardinality-estimation-best-practices.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/abo-optimizer/intelligent-cardinality-estimation/intelligent-cardinality-estimation-overview.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/abo-optimizer/intelligent-cardinality-estimation/intelligent-cardinality-estimation-overview.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/abo-optimizer/intelligent-cardinality-estimation/intelligent-cardinality-estimation-overview.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/abo-optimizer/intelligent-cardinality-estimation/intelligent-cardinality-estimation-overview.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/abo-optimizer/intelligent-cardinality-estimation/intelligent-cardinality-estimation-prerequisites.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/abo-optimizer/intelligent-cardinality-estimation/intelligent-cardinality-estimation-prerequisites.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/abo-optimizer/intelligent-cardinality-estimation/intelligent-cardinality-estimation-prerequisites.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/abo-optimizer/intelligent-cardinality-estimation/intelligent-cardinality-estimation-prerequisites.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/abo-optimizer/intelligent-cardinality-estimation/intelligent-cardinality-estimation-troubleshooting.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/abo-optimizer/intelligent-cardinality-estimation/intelligent-cardinality-estimation-troubleshooting.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/abo-optimizer/intelligent-cardinality-estimation/intelligent-cardinality-estimation-troubleshooting.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/abo-optimizer/intelligent-cardinality-estimation/intelligent-cardinality-estimation-troubleshooting.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/abo-optimizer/intelligent-cardinality-estimation/intelligent-cardinality-estimation-usage-guide.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/abo-optimizer/intelligent-cardinality-estimation/intelligent-cardinality-estimation-usage-guide.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/abo-optimizer/intelligent-cardinality-estimation/intelligent-cardinality-estimation-usage-guide.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/abo-optimizer/intelligent-cardinality-estimation/intelligent-cardinality-estimation-usage-guide.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/1-x-tuner-parameter-optimization-and-diagnosis/1-4-obtaining-help-information.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/1-x-tuner-parameter-optimization-and-diagnosis/1-4-obtaining-help-information.md deleted file mode 100644 index 12f32ef275085145a39b65474fc5e50fb90cb69e..0000000000000000000000000000000000000000 --- a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/1-x-tuner-parameter-optimization-and-diagnosis/1-4-obtaining-help-information.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -title: Obtaining Help Information -summary: Obtaining Help Information -author: Guo Huan -date: 2021-05-19 ---- - -# Obtaining Help Information - -Before starting the tuning program, run the following command to obtain help information: - -```bash -python main.py --help -``` - -The command output is as follows: - -```bash -usage: main.py [-h] [-m {train,tune}] [-f CONFIG_FILE] [--db-name DB_NAME] -[--db-user DB_USER] [--port PORT] [--host HOST] -[--host-user HOST_USER] [--host-ssh-port HOST_SSH_PORT] -[--scenario {ap,tp,htap}] [--benchmark BENCHMARK] -[--model-path MODEL_PATH] [-v] - -X-Tuner: a self-tuning toolkit for MogD. - -optional arguments: --h, --help show this help message and exit --m {train,tune}, --mode {train,tune} -train a reinforcement learning model or tune by your -trained model. --f CONFIG_FILE, --config-file CONFIG_FILE -you can pass a config file path or you should manually -set database information. ---db-name DB_NAME database name. ---db-user DB_USER database user name. ---port PORT database connection port. ---host HOST where did your database install on? ---host-user HOST_USER -user name of the host where your database installed -on. ---host-ssh-port HOST_SSH_PORT -host ssh port. ---scenario {ap,tp,htap} ---benchmark BENCHMARK ---model-path MODEL_PATH -the place where you want to save model weights to or -load model weights from. --v, --version -show version. -``` diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/4-forcast-trend-prediction/4-4-obtaining-help-information.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/4-forcast-trend-prediction/4-4-obtaining-help-information.md deleted file mode 100644 index cfb64369f6f062cd519b78da1688f2fa4073a53b..0000000000000000000000000000000000000000 --- a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/4-forcast-trend-prediction/4-4-obtaining-help-information.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -title: Obtaining Help Information -summary: Obtaining Help Information -author: Guo Huan -date: 2022-05-06 ---- - -# Obtaining Help Information - -You can run the **–help** command to obtain the help information. For example: - -``` -gs_dbmind component forecast --help -``` - -``` -usage: [-h] -c DIRECTORY [--metric-name METRIC_NAME] [--host HOST] [--start-time TIMESTAMP_IN_MICROSECONDS] [--end-time TIMESTAMP_IN_MICROSECONDS] [--retention-days DAYS] - {show,clean} - -Workload Forecasting: Forecast monitoring metrics - -positional arguments: - {show,clean} choose a functionality to perform - -optional arguments: - -h, --help show this help message and exit - -c DIRECTORY, --conf DIRECTORY - set the directory of configuration files - --metric-name METRIC_NAME - set a metric name you want to retrieve - --host HOST set a host you want to retrieve - --start-time TIMESTAMP_IN_MICROSECONDS - set a start time of for retrieving - --end-time TIMESTAMP_IN_MICROSECONDS - set a end time of for retrieving - --retention-days DAYS - clear historical diagnosis results and set the maximum number of days to retain data -``` diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/4-forcast-trend-prediction/4-5-command-reference.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/4-forcast-trend-prediction/4-5-command-reference.md deleted file mode 100644 index 4d7741bdbff00e2cb915706066216ec1358b5cef..0000000000000000000000000000000000000000 --- a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/4-forcast-trend-prediction/4-5-command-reference.md +++ /dev/null @@ -1,21 +0,0 @@ ---- -title: Command Reference -summary: Command Reference -author: Guo Huan -date: 2022-05-06 ---- - -# Command Reference - -**Table 1** gs_dbmind component forecast parameters - -| Parameter | Description | Value Range | -| :-------------- | :----------------------------------------------------------- | :----------------------------------------------------------- | -| -h, –help | Help information | - | -| action | Action parameter | - **show**: displays results.
- **clean**: clears results. | -| -c, –conf | Configuration directory | - | -| –metric-name | Specifies the metric name to be displayed, which is used for filtering. | - | -| –host | Specifies the service IP address and port number, which is used for filtering. | - | -| –start-time | Timestamp of the start time, in milliseconds. Alternatively, the date and time format is %Y-%m-%d %H:%M:%S. | Positive integer or date and time format | -| –end-time | Timestamp of the end time, in milliseconds. Alternatively, the date and time format is %Y-%m-%d %H:%M:%S. | Positive integer or date and time format | -| –retention-days | Number of days retaining results | Non-negative real number | diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/ai-sub-functions-of-the-dbmind.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/ai-sub-functions-of-the-dbmind.md new file mode 100644 index 0000000000000000000000000000000000000000..a787f48203ae478db1f36f100d59213bfba9d612 --- /dev/null +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/ai-sub-functions-of-the-dbmind.md @@ -0,0 +1,19 @@ +--- +title: AI Sub-Functions of the DBMind +summary: AI Sub-Functions of the DBMind +author: zhang cuiping +date: 2023-04-07 +--- + +# AI Sub-Functions of the DBMind + +You can run the **component** subcommand of **gs_dbmind** to enable the corresponding AI sub-functions. The following sections describe the AI functions in detail. + +- **[X-Tuner: Parameter Tuning and Diagnosis](./x-tuner-parameter-optimization-and-diagnosis/x-tuner-parameter-optimization-and-diagnosis.md)** +- **[Index-advisor: Index Recommendation](./index-advisor-index-recommendation/index-advisor-index-recommendation.md)** +- **[Slow Query Diagnosis: Root Cause Analysis for Slow SQL Statements](./slow-query-diagnosis-root-cause-analysis-for-slow-sql-statements/slow-sql-statements.md)** +- **[Forecast: Trend Prediction](./forcast-trend-prediction/forcast-trend-prediction.md)** +- **[SQLdiag: Slow SQL Discovery](./sqldiag-slow-sql-discovery/sqldiag-slow-sql-discovery.md)** +- **[SQL Rewriter: SQL Statement Rewriting](./sql-rewriter-sql-statement-rewriting/sql-rewriter-sql-statement-rewriting.md)** +- **[Anomaly Detection](./anomaly-detection/anomaly-detection.md)** +- **[Anomaly-analysis: Multi-Metric Correlation Analysis](./anomaly-analysis/anomaly-analysis.md)** \ No newline at end of file diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-analysis/anomaly-analysis-command-reference.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-analysis/anomaly-analysis-command-reference.md new file mode 100644 index 0000000000000000000000000000000000000000..f9a438d81d56a337d8ba544710e577c2cc7bb293 --- /dev/null +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-analysis/anomaly-analysis-command-reference.md @@ -0,0 +1,20 @@ +--- +title: Command Reference +summary: Command Reference +author: zhang cuiping +date: 2023-04-07 +--- + +# Command Reference + +**Table 1** Command line parameters + +| Parameter | Description | Value Range | +| :--------------- | :----------------------------------------------------------- | :--------------------------------------- | +| -h, --help | Help command | - | +| -c, --conf | Configuration file directory | - | +| -m, --metric | Metric name to be displayed | - | +| -H, --host | Data source IP address which is used to filter data | IP address or IP address + port number | +| -s, --start-time | Timestamp of the start time, in milliseconds. Alternatively, the date and time format is %Y-%m-%d %H:%M:%S. | Positive integer or date and time format | +| -e, --end-time | Timestamp of the start time, in milliseconds. Alternatively, the date and time format is %Y-%m-%d %H:%M:%S. | Positive integer or date and time format | +| --csv-dump-path | Path of the exported CSV file. | - | \ No newline at end of file diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-analysis/anomaly-analysis-obtaining-help-information.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-analysis/anomaly-analysis-obtaining-help-information.md new file mode 100644 index 0000000000000000000000000000000000000000..8a0b4ae1c8d65344c3b05bf8ef573dbd91df0800 --- /dev/null +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-analysis/anomaly-analysis-obtaining-help-information.md @@ -0,0 +1,39 @@ +--- +title: Obtaining Help Information +summary: Obtaining Help Information +author: zhang cuiping +date: 2023-04-07 +--- + +# Obtaining Help Information + +You can run the **–help** command to obtain the help information. For example: + +``` +gs_dbmind component anomaly_detection --help +``` + +The following information is displayed: + +``` +usage: anomaly_analysis.py [-h] -c CONF -m METRIC -s START_TIME -e END_TIME -H + HOST [--csv-dump-path CSV_DUMP_PATH] + +Workload Anomaly analysis: Anomaly analysis of monitored metric. + +optional arguments: + -h, --help show this help message and exit + -c CONF, --conf CONF set the directory of configuration files + -m METRIC, --metric METRIC + set the metric name you want to retrieve + -s START_TIME, --start-time START_TIME + set the start time of for retrieving in ms, supporting + UNIX-timestamp with microsecond or datetime format + -e END_TIME, --end-time END_TIME + set the end time of for retrieving in ms, supporting + UNIX-timestamp with microsecond or datetime format + -H HOST, --host HOST set a host of the metric, ip only or ip and port. + --csv-dump-path CSV_DUMP_PATH + dump the result csv file to the dump path if it is + specified. +``` \ No newline at end of file diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-analysis/anomaly-analysis-overview.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-analysis/anomaly-analysis-overview.md new file mode 100644 index 0000000000000000000000000000000000000000..2d38ae1301ce22e97f520151c144198b7196d108 --- /dev/null +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-analysis/anomaly-analysis-overview.md @@ -0,0 +1,10 @@ +--- +title: Overview +summary: Overview +author: zhang cuiping +date: 2023-04-07 +--- + +# Overview + +The Anomaly analysis multi-metric correlation module analyzes the Pearson correlation coefficient of time series data to find the metrics that are most closely related to known exceptions. The framework of this module is decoupled. The supported time series databases include Prometheus and InfluxDB. diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-analysis/anomaly-analysis-troubleshooting.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-analysis/anomaly-analysis-troubleshooting.md new file mode 100644 index 0000000000000000000000000000000000000000..6bbef3ce046ff16424cc90fb08cf2bddcafc851d --- /dev/null +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-analysis/anomaly-analysis-troubleshooting.md @@ -0,0 +1,10 @@ +--- +title: Troubleshooting +summary: Troubleshooting +author: zhang cuiping +date: 2023-04-07 +--- + +# Troubleshooting + +Analysis scenario failure: Check whether the configuration file path is correct and whether the configuration file information is complete. Check whether the metric name is correct, whether the host address is correct, and whether the metric data exists in the start time and end time. \ No newline at end of file diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-analysis/anomaly-analysis-usage-guide.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-analysis/anomaly-analysis-usage-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..3e9b455afaf6bcd5415a12c92fc8e0a716986acf --- /dev/null +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-analysis/anomaly-analysis-usage-guide.md @@ -0,0 +1,25 @@ +--- +title: Usage Guide +summary: Usage Guide +author: zhang cuiping +date: 2023-04-07 +--- + +# Usage Guide + +Assume that the metric collection system is running properly and the configuration file directory **confpath** has been initialized. You can run the following command to implement this feature: + +For a metric, analyze the correlation between other metrics and the data of the metric from timestamps1 to timestamps2 on a specific node. + +``` +gs_dbmind component anomaly_analysis --conf confpath --metric metric_name --start-time timestamps1 --end-time timestamps2 --host ip_address +``` + +For a metric, analyze the correlation between other metrics and the metric data from timestamps1 to timestamps2 on a specific node, and save the analysis result as a CSV file. + +``` +gs_dbmind component anomaly_analysis --conf confpath --metric metric_name --start-time timestamps1 --end-time timestamps2 --host ip_address --csv-dump-path csv_path +``` + +> ![img](https://cdn-mogdb.enmotech.com/docs-media/icon/icon-note.gif) **NOTE:** When setting anomaly_analysis parameters, ensure that start-time is at least 30 seconds earlier than end-time. +> \ No newline at end of file diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-analysis/anomaly-analysis.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-analysis/anomaly-analysis.md new file mode 100644 index 0000000000000000000000000000000000000000..e226f720fb646e01d69f983f788f29ab16a00bf7 --- /dev/null +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-analysis/anomaly-analysis.md @@ -0,0 +1,14 @@ +--- +title: Anomaly-analysis +summary: Anomaly-analysis +author: zhang cuiping +date: 2023-04-07 +--- + +# Anomaly-analysis: Multi-Metric Correlation Analysis + +- **[Overview](anomaly-analysis-overview.md)** +- **[Usage Guide](anomaly-analysis-usage-guide.md)** +- **[Obtaining Help Information](anomaly-analysis-obtaining-help-information.md)** +- **[Command Reference](anomaly-analysis-command-reference.md)** +- **[Troubleshooting](anomaly-analysis-troubleshooting.md)** \ No newline at end of file diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/7-anomaly-detection/7-4-command-reference.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-detection/anomaly-detection-command-reference.md similarity index 50% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/7-anomaly-detection/7-4-command-reference.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-detection/anomaly-detection-command-reference.md index 5d966981e245efb9d5f7683dd79a78ff24a1d92d..1f42722e115274c0ed12eeede2feb3f7c08e7571 100644 --- a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/7-anomaly-detection/7-4-command-reference.md +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-detection/anomaly-detection-command-reference.md @@ -9,13 +9,13 @@ date: 2022-10-24 **Table 1** Command line parameters -| Parameter | Description | Value Range | -| :---------------- | :----------------------------------------------------------- | :------------------------------------------- | -| -h, --help | Help command | - | -| --action | Action parameter | - **overview**
- **plot**: visualization | -| -c，--conf | Configuration file directory | - | -| -m，--metric-name | Metric name to be displayed | - | -| -H, --host | Data source IP address which is used to filter data | IP address or IP address + port number | -| -a, --anomaly | Anomaly detection mode, which is used for filtering | - | -| -s, --start-time | Timestamp of the start time, in milliseconds. Alternatively, the date and time format is %Y-%m-%d %H:%M:%S. | Positive integer or date and time format | -| -e, --end-time | Timestamp of the start time, in milliseconds. Alternatively, the date and time format is %Y-%m-%d %H:%M:%S. | Positive integer or date and time format | +| Parameter | Description | Value Range | +| :---------------- | :----------------------------------------------------------- | :--------------------------------------- | +| -h, --help | Help command | - | +| --action | Action parameter | **overview****plot**: visualization | +| -c, --conf | Configuration file directory | - | +| -m, --metric-name | Metric name to be displayed | - | +| -H, --host | Data source IP address which is used to filter data | IP address or IP address + port number | +| -a, --anomaly | Anomaly detection mode, which is used for filtering | - | +| -s, --start-time | Timestamp of the start time, in milliseconds. Alternatively, the date and time format is %Y-%m-%d %H:%M:%S. | Positive integer or date and time format | +| -e, --end-time | Timestamp of the start time, in milliseconds. Alternatively, the date and time format is %Y-%m-%d %H:%M:%S. | Positive integer or date and time format | \ No newline at end of file diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/7-anomaly-detection/7-3-obtaining-help-information.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-detection/anomaly-detection-obtaining-help-information.md similarity index 94% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/7-anomaly-detection/7-3-obtaining-help-information.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-detection/anomaly-detection-obtaining-help-information.md index fe9935023cfb1849c615f816285a4934204633ab..bd14d90b9e58bbbfb2dc1902df94c79772c4ed1b 100644 --- a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/7-anomaly-detection/7-3-obtaining-help-information.md +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-detection/anomaly-detection-obtaining-help-information.md @@ -7,7 +7,7 @@ date: 2022-10-24 # Obtaining Help Information -You can run the **--help** command to obtain the help information. For example: +You can run the **–help** command to obtain the help information. For example: ```shell gs_dbmind component anomaly_detection --help diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/7-anomaly-detection/7-1-overview.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-detection/anomaly-detection-overview.md similarity index 98% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/7-anomaly-detection/7-1-overview.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-detection/anomaly-detection-overview.md index d2fb9557749288d087305f2d376380e195c780bd..fe7eaf71dda3f7ea8516e9d5d781017749e01ead 100644 --- a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/7-anomaly-detection/7-1-overview.md +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-detection/anomaly-detection-overview.md @@ -7,4 +7,4 @@ date: 2022-10-24 # Overview -The anomaly detection module implements time series data based on statistics methods to detect possible exceptions in the data. The framework of this module is decoupled to flexibly replace different anomaly detection algorithms. In addition, this module can automatically select algorithms based on different features of time series data. It supports anomaly value detection, threshold detection, box plot detection, gradient detection, growth rate detection, fluctuation rate detection, and status conversion detection. +The anomaly detection module implements time series data based on statistics methods to detect possible exceptions in the data. The framework of this module is decoupled to flexibly replace different anomaly detection algorithms. In addition, this module can automatically select algorithms based on different features of time series data. It supports anomaly value detection, threshold detection, box plot detection, gradient detection, growth rate detection, fluctuation rate detection, and status conversion detection. \ No newline at end of file diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/7-anomaly-detection/7-5-troubleshooting.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-detection/anomaly-detection-troubleshooting.md similarity index 97% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/7-anomaly-detection/7-5-troubleshooting.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-detection/anomaly-detection-troubleshooting.md index 5683b12cd0e85f26b80671583f0fe52b91d64702..05e297d6256fecfea76576cdf1b4c1ae662ac9ae 100644 --- a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/7-anomaly-detection/7-5-troubleshooting.md +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-detection/anomaly-detection-troubleshooting.md @@ -8,4 +8,4 @@ date: 2022-10-24 # Troubleshooting - Overview scenario failure: Check whether the configuration file path is correct and whether the configuration file information is complete. Check whether the metric name is correct, whether the host IP address is correct, whether the anomaly detection type is correct, and whether the metric data exists in the start time and end time. -- Visualization scenario failure: Check whether the configuration file path is correct and whether the configuration file information is complete. Check whether the metric name is correct, whether the host IP address is correct, whether the anomaly detection type is correct, and whether the metric data exists in the start time and end time. +- Visualization scenario failure: Check whether the configuration file path is correct and whether the configuration file information is complete. Check whether the metric name is correct, whether the host IP address is correct, whether the anomaly detection type is correct, and whether the metric data exists in the start time and end time. \ No newline at end of file diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/7-anomaly-detection/7-2-usage-guide.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-detection/anomaly-detection-usage-guide.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/7-anomaly-detection/7-2-usage-guide.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-detection/anomaly-detection-usage-guide.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-detection/anomaly-detection.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-detection/anomaly-detection.md new file mode 100644 index 0000000000000000000000000000000000000000..1974c16f3fe145fadda9099a1004e1c6c300ae34 --- /dev/null +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-detection/anomaly-detection.md @@ -0,0 +1,14 @@ +--- +title: Anomaly detection +summary: Anomaly detection +author: zhang cuiping +date: 2023-04-07 +--- + +# Anomaly Detection + +- **[Overview](anomaly-detection-overview.md)** +- **[Usage Guide](anomaly-detection-usage-guide.md)** +- **[Obtaining Help Information](anomaly-detection-obtaining-help-information.md)** +- **[Command Reference](anomaly-detection-command-reference.md)** +- **[Troubleshooting](anomaly-detection-troubleshooting.md)** \ No newline at end of file diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/forcast-trend-prediction/forcast-trend-prediction-command-reference.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/forcast-trend-prediction/forcast-trend-prediction-command-reference.md new file mode 100644 index 0000000000000000000000000000000000000000..da351b201189e32ecedfbc771d8b2d5542699089 --- /dev/null +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/forcast-trend-prediction/forcast-trend-prediction-command-reference.md @@ -0,0 +1,21 @@ +--- +title: Command Reference +summary: Command Reference +author: Guo Huan +date: 2022-05-06 +--- + +# Command Reference + +**Table 1** gs_dbmind component forecast parameters + +| Parameter | Description | Value Range | +| :--------------- | :----------------------------------------------------------- | :----------------------------------------------------------- | +| -h, --help | Help information | - | +| action | Action parameter | **show**: displays results.**clean**: clears results. | +| -c, --conf | Configuration directory | - | +| --metric-name | Specifies the metric name to be displayed, which is used for filtering. | - | +| --host | Specifies the service IP address and port number, which is used for filtering. | - | +| --start-time | Timestamp of the start time, in milliseconds. Alternatively, the date and time format is %Y-%m-%d %H:%M:%S. | Positive integer or date and time format | +| --end-time | Timestamp of the start time, in milliseconds. Alternatively, the date and time format is %Y-%m-%d %H:%M:%S. | Timestamp of the start time, in milliseconds. Alternatively, the date and time format is %Y-%m-%d %H:%M:%S. | +| --retention-days | Number of days retaining results | Non-negative real number | \ No newline at end of file diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/4-forcast-trend-prediction/4-2-environment-deployment.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/forcast-trend-prediction/forcast-trend-prediction-environment-deployment.md similarity index 72% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/4-forcast-trend-prediction/4-2-environment-deployment.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/forcast-trend-prediction/forcast-trend-prediction-environment-deployment.md index e370be78bafde7140d95a7e9ec3ffb77de4d6a94..d3f1eae47b08d4eb3f080a86574727661abc583e 100644 --- a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/4-forcast-trend-prediction/4-2-environment-deployment.md +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/forcast-trend-prediction/forcast-trend-prediction-environment-deployment.md @@ -7,4 +7,4 @@ date: 2022-05-06 # Environment Deployment -The metric collection system is running properly. +The metric collection system is running properly. \ No newline at end of file diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/forcast-trend-prediction/forcast-trend-prediction-obtaining-help-information.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/forcast-trend-prediction/forcast-trend-prediction-obtaining-help-information.md new file mode 100644 index 0000000000000000000000000000000000000000..bb0efd0c815fddccc53f0af19435029b9639aaed --- /dev/null +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/forcast-trend-prediction/forcast-trend-prediction-obtaining-help-information.md @@ -0,0 +1,57 @@ +--- +title: Obtaining Help Information +summary: Obtaining Help Information +author: Guo Huan +date: 2022-05-06 +--- + +# Obtaining Help Information + +You can run the **–help** command to obtain the help information. For example: + +``` +gs_dbmind component forecast --help +``` + +``` +usage: forecast.py [-h] -c DIRECTORY [--metric-name METRIC_NAME] [--host HOST] + [--labels LABELS] [--start-time TIMESTAMP_IN_MICROSECONDS] + [--end-time TIMESTAMP_IN_MICROSECONDS] + [--retention-days DAYS] [--upper UPPER] [--lower LOWER] + [--warning-hours WARNING-HOURS] + [--csv-dump-path CSV_DUMP_PATH] + {show,clean,early-warning} + +Workload Forecasting: Forecast monitoring metrics + +positional arguments: + {show,clean,early-warning} + Choose a functionality to perform + +optional arguments: + -h, --help show this help message and exit + -c DIRECTORY, --conf DIRECTORY + Set the directory of configuration files + --metric-name METRIC_NAME + Set a metric name you want to retrieve + --host HOST Set a host you want to retrieve. IP only or IP with + port. + --labels LABELS A list of label (format is label=name) separated by + comma(,). Using in warning. + --start-time TIMESTAMP_IN_MICROSECONDS + Set a start time for retrieving, supporting UNIX- + timestamp with microsecond or datetime format + --end-time TIMESTAMP_IN_MICROSECONDS + Set an end time for retrieving, supporting UNIX- + timestamp with microsecond or datetime format + --retention-days DAYS + Clear historical diagnosis results and set the maximum + number of days to retain data + --upper UPPER The upper value of early-warning. Using in warning. + --lower LOWER The lower value of early-warning. Using in warning. + --warning-hours WARNING-HOURS + warning length, unit is hour. + --csv-dump-path CSV_DUMP_PATH + Dump the result CSV file to the path if it is + specified. Use in warning. +``` diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/4-forcast-trend-prediction/4-1-overview.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/forcast-trend-prediction/forcast-trend-prediction-overview.md similarity index 93% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/4-forcast-trend-prediction/4-1-overview.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/forcast-trend-prediction/forcast-trend-prediction-overview.md index 33df1fa40e3f83dc181222683c8304ec1026fb7a..b4a487d09f4db62acc320c9dec4846297a5e7cb0 100644 --- a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/4-forcast-trend-prediction/4-1-overview.md +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/forcast-trend-prediction/forcast-trend-prediction-overview.md @@ -7,4 +7,4 @@ date: 2022-05-06 # Overview -The trend prediction module predicts the future time series change trend based on historical time series data. The framework of this module has been decoupled to flexibly change prediction algorithms. This module can automatically select algorithms for different feature time series. The LR regression algorithm for linear feature time series prediction and the ARIMA algorithm for non-linear feature prediction are supported. At present, this module can cover the accurate prediction of linear time series, non-linear time series and periodic time series. +The trend prediction module predicts the future time series change trend based on historical time series data. The framework of this module has been decoupled to flexibly change prediction algorithms. This module can automatically select algorithms for different feature time series. The LR regression algorithm for linear feature time series prediction and the ARIMA algorithm for non-linear feature prediction are supported. At present, this module can cover the accurate prediction of linear time series, non-linear time series and periodic time series. \ No newline at end of file diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/4-forcast-trend-prediction/4-6-troubleshooting.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/forcast-trend-prediction/forcast-trend-prediction-troubleshooting.md similarity index 99% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/4-forcast-trend-prediction/4-6-troubleshooting.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/forcast-trend-prediction/forcast-trend-prediction-troubleshooting.md index 46cf947bc059d7181b2f2921c8611c7368d95d05..9af967f6a9cf7547f7780a68f456f0ffb7d20206 100644 --- a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/4-forcast-trend-prediction/4-6-troubleshooting.md +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/forcast-trend-prediction/forcast-trend-prediction-troubleshooting.md @@ -8,4 +8,4 @@ date: 2022-05-06 # Troubleshooting - Considering the actual service and model prediction effect, you are advised to set the trend prediction duration to a value greater than 3600 seconds. (If the metric collection period is 15 seconds, the number of data records collected is 240.) Otherwise, the prediction effect will deteriorate, and the service will be abnormal when the data volume is extremely small. The default value is 3600 seconds. -- After the parameters in the configuration file are reset, you need to restart the service process for the settings to take effect. +- After the parameters in the configuration file are reset, you need to restart the service process for the settings to take effect. \ No newline at end of file diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/4-forcast-trend-prediction/4-3-usage-guide.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/forcast-trend-prediction/forcast-trend-prediction-usage-guide.md similarity index 99% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/4-forcast-trend-prediction/4-3-usage-guide.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/forcast-trend-prediction/forcast-trend-prediction-usage-guide.md index d5283c1ef520a3e9128f9d8f79688268bac1d939..c561ba0a138b3fca4f72ef99833d8514ae9182d7 100644 --- a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/4-forcast-trend-prediction/4-3-usage-guide.md +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/forcast-trend-prediction/forcast-trend-prediction-usage-guide.md @@ -37,4 +37,4 @@ Assume that the **confpath** configuration file directory has been initialized. ``` gs_dbmind service stop -c confpath - ``` \ No newline at end of file + ``` diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/forcast-trend-prediction/forcast-trend-prediction.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/forcast-trend-prediction/forcast-trend-prediction.md new file mode 100644 index 0000000000000000000000000000000000000000..4eba74876d68e1dad23c87f358b8f6490536bd4f --- /dev/null +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/forcast-trend-prediction/forcast-trend-prediction.md @@ -0,0 +1,15 @@ +--- +title: Forecast +summary: Forecast +author: zhang cuiping +date: 2023-04-07 +--- + +# Forecast + +- **[Overview](forcast-trend-prediction-overview.md)** +- **[Environment Deployment](forcast-trend-prediction-environment-deployment.md)** +- **[Usage Guide](forcast-trend-prediction-usage-guide.md)** +- **[Obtaining Help Information](forcast-trend-prediction-obtaining-help-information.md)** +- **[Command Reference](forcast-trend-prediction-command-reference.md)** +- **[Troubleshooting](forcast-trend-prediction-troubleshooting.md)** \ No newline at end of file diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/index-advisor-index-recommendation/index-advisor-index-recommendation.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/index-advisor-index-recommendation/index-advisor-index-recommendation.md new file mode 100644 index 0000000000000000000000000000000000000000..62ae5c9e63b0230a2a224c09bb91d433653dc446 --- /dev/null +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/index-advisor-index-recommendation/index-advisor-index-recommendation.md @@ -0,0 +1,14 @@ +--- +title: Index-advisor +summary: Index-advisor +author: zhang cuiping +date: 2023-04-07 +--- + +# Index-advisor: Index Recommendation + +This section describes the index recommendation functions, including single-query index recommendation, virtual index recommendation, and workload-level index recommendation. + +- **[Single-query Index Recommendation](single-query-index-recommendation.md)** +- **[Virtual Index](virtual-index.md)** +- **[Workload-level Index Recommendation](workload-level-index-recommendation.md)** \ No newline at end of file diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/2-index-advisor-index-recommendation/2-1-single-query-index-recommendation.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/index-advisor-index-recommendation/single-query-index-recommendation.md similarity index 81% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/2-index-advisor-index-recommendation/2-1-single-query-index-recommendation.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/index-advisor-index-recommendation/single-query-index-recommendation.md index ea47de36d76b6a67f064224df74ff0668de2e638..4a3526f927ce4e2c30bc3eb5a6c263c982bbef4e 100644 --- a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/2-index-advisor-index-recommendation/2-1-single-query-index-recommendation.md +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/index-advisor-index-recommendation/single-query-index-recommendation.md @@ -9,7 +9,7 @@ date: 2021-05-19 The single-query index recommendation function allows users to directly perform operations in the database. This function generates recommended indexes for a single query statement entered by users based on the semantic information of the query statement and the statistics of the database. This function involves the following interfaces: -**Table 1** Single-query index recommendation interfaces +**Table 1** Single-query index recommendation APIs | Function Name | Parameter | Description | | :-------------- | :------------------- | :----------------------------------------------------------- | @@ -18,7 +18,9 @@ The single-query index recommendation function allows users to directly perform > ![img](https://cdn-mogdb.enmotech.com/docs-media/icon/icon-note.gif) **NOTE:** > > - This function supports only a single SELECT statement and does not support other types of SQL statements. -> - Partitioned tables, column-store tables, segment-paged tables, common views, materialized views, global temporary tables, and encrypted databases are not supported. +> - Column-store tables, segment-paged tables, common views, materialized views, global temporary tables, and encrypted databases are not supported. + +
## Application Scenarios @@ -43,7 +45,7 @@ CREATE INDEX idx on bmsql_customer(c_w_id); Some SQL statements may also be recommended to create a join index, for example: ```sql -mogdb=# select "table", "column" from gs_index_advise('select name, age, sex from t1 where age >= 18 and age < 35 and sex = ''f'';'); +MogDB=# select "table", "column" from gs_index_advise('select name, age, sex from t1 where age >= 18 and age < 35 and sex = ''f'';'); table | column -------+------------ t1 | (age, sex) @@ -56,4 +58,14 @@ The preceding statement indicates that a join index **(age, sex)** needs to be c CREATE INDEX idx1 on t1(age, sex); ``` +You can recommend specific index types for partitioned tables. For example: + +```sql +MogDB=# select "table", "column", "indextype" from gs_index_advise('select name, age, sex from range_table where age = 20;'); + table | column | indextype +-------+--------+----------- + t1 | age | global +(1 row) +``` + > ![img](https://cdn-mogdb.enmotech.com/docs-media/icon/icon-note.gif) **NOTE:** Parameters of the system function **gs_index_advise()** are of the text type. If the parameters contain special characters such as single quotation marks ('), you can use single quotation marks (') to escape the special characters. For details, see the preceding example. diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/2-index-advisor-index-recommendation/2-2-virtual-index.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/index-advisor-index-recommendation/virtual-index.md similarity index 98% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/2-index-advisor-index-recommendation/2-2-virtual-index.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/index-advisor-index-recommendation/virtual-index.md index 4d6cd4168804fde4c3b17cedc26d968087243ba0..e248174e4d32775a98532c04eab18e6a80bfa747 100644 --- a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/2-index-advisor-index-recommendation/2-2-virtual-index.md +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/index-advisor-index-recommendation/virtual-index.md @@ -9,9 +9,9 @@ date: 2021-05-19 The virtual index function allows users to directly perform operations in the database. This function simulates the creation of a real index to avoid the time and space overhead required for creating a real index. Based on the virtual index, users can evaluate the impact of the index on the specified query statement by using the optimizer. -This function involves the following interfaces: +This function involves the following APIs: -**Table 1** Virtual index function interfaces +**Table 1** Virtual index function APIs | Function Name | Parameter | Description | | :------------------- | :------------------------------------------------------ | :----------------------------------------------------------- | @@ -29,6 +29,8 @@ This function involves the following GUC parameters: | :---------------- | :-------------------------------------------- | :------------ | | enable_hypo_index | Whether to enable the virtual index function. | off | +
+ ## Procedure 1. Use the **hypopg_create_index** function to create a virtual index. For example: @@ -112,7 +114,7 @@ This function involves the following GUC parameters: mogdb=> select * from hypopg_reset_index(); hypopg_reset_index -------------------- - + (1 row) ``` diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/2-index-advisor-index-recommendation/2-3-workload-level-index-recommendation.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/index-advisor-index-recommendation/workload-level-index-recommendation.md similarity index 41% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/2-index-advisor-index-recommendation/2-3-workload-level-index-recommendation.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/index-advisor-index-recommendation/workload-level-index-recommendation.md index 409d1c1643699a00f5cef4b611b98d4d8c5e07cd..7d204ae7d3159f4b3d2b988c31ec70be2d0d528c 100644 --- a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/2-index-advisor-index-recommendation/2-3-workload-level-index-recommendation.md +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/index-advisor-index-recommendation/workload-level-index-recommendation.md @@ -12,84 +12,116 @@ For workload-level indexes, you can run scripts outside the database to use this ## Prerequisites - The database is normal, and the client can be connected properly. +- The **gsql** tool has been installed by the current user, and the tool path has been added to the _PATH_environment variable. -- The **gsql** tool has been installed by the current user, and the tool path has been added to the **PATH** environment variable. +## Service Data Extraction -- The Python 3.6+ environment is available. +### SQL in Logs -- To use the service data extraction function, you need to set the GUC parameters of the node whose data is to be collected as follows: +1. Set the GUC parameters. + log_min_duration_statement = 0 + log_statement= 'all' - - log_min_duration_statement = 0 - - - log_statement= 'all' - - > ![img](https://cdn-mogdb.enmotech.com/docs-media/icon/icon-note.gif) **NOTE:** After service data extraction is complete, you are advised to restore the preceding GUC parameters. Otherwise, log files may be expanded. - -## Procedure for Using the Service Data Extraction Script - -1. Set the GUC parameters according to instructions in the prerequisites. - -2. Run the Python script **extract_log.py**: +2. Run the following command to extract SQL statements based on logs: ``` - ython extract_log.py [l LOG_DIRECTORY] [f OUTPUT_FILE] [-d DATABASE] [-U USERNAME][--start_time] [--sql_amount] [--statement] [--json] + gs_dbmind component extract_log [l LOG_DIRECTORY] [f OUTPUT_FILE] [p LOG_LINE_PREFIX] [-d DATABASE] [-U USERNAME][--start_time] [--sql_amount] [--statement] [--json] [--max_reserved_period] [--max_template_num] ``` The input parameters are as follows: - **LOG_DIRECTORY**: directory for storing **pg_log**. - **OUTPUT_PATH**: path for storing the output SQL statements, that is, path for storing the extracted service data. + - **LOG_LINE_PREFIX**: specifies the prefix format of each log. - **DATABASE** (optional): database name. If this parameter is not specified, all databases are selected by default. - **USERNAME** (optional): username. If this parameter is not specified, all users are selected by default. - **start_time** (optional): start time for log collection. If this parameter is not specified, all files are collected by default. - **sql_amount** (optional): maximum number of SQL statements to be collected. If this parameter is not specified, all SQL statements are collected by default. - **statement** (optional): Collects the SQL statements starting with **statement** in **pg_log log**. If this parameter is not specified, the SQL statements are not collected by default. - - **json**: Specifies that the collected log files are stored in JSON format after SQL normalization. If the default format is not specified, each SQL statement occupies a line. + - **json** (optional): specifies that the collected log files are stored in JSON format after SQL normalization. If no format is specified, each SQL statement occupies a line. + - **max_reserved_period** (optional): specifies the maximum number of days of reserving the template in incremental log collection in JSON mode. If this parameter is not specified, the template is reserved by default. The unit is day. + - **max_template_num** (optional): Specifies the maximum number of templates that can be reserved in JSON mode. If this parameter is not specified, all templates are reserved by default. An example is provided as follows. ``` - python extract_log.py $GAUSSLOG/pg_log/dn_6001 sql_log.txt -d postgres -U omm --start_time '2021-07-06 00:00:00' --statement + gs_dbmind component extract_log $GAUSSLOG/pg_log/dn_6001 sql_log.txt '%m %c %d %p %a %x %n %e' -d postgres -U omm --start_time '2021-07-06 00:00:00' --statement ``` -3. Change the GUC parameter values set in step 1 to the values before the setting. + > ![img](https://cdn-mogdb.enmotech.com/docs-media/icon/icon-note.gif) **NOTE:** If the **-d/-U** parameter is specified, the prefix of each log record must contain **%d** and **%u**. If transactions need to be extracted, **%p** must be specified. For details, see the **log_line_prefix** parameter. It is recommended that the value of **max_template_num** be less than or equal to **5000** to avoid long execution time of workload indexes. + > + +3. Change the GUC parameter values set in [1](#1) to the values before the setting. + + > ![img](https://cdn-mogdb.enmotech.com/docs-media/icon/icon-note.gif) **NOTE:** + > + > After service data extraction is completed, restore the GUC parameters to the default configurations. Otherwise, it may lead to log file bloating. + +### SQL in System Tables + +To enable this function, run the following command: + +``` +echo PASSWORD | gs_dbmind component fetch_statement [DB_PORT] [DATABASE] [OUTPUT] [--db-host DB_HOST] [-U DB_USER] [--schema SCHEMA] +[--statement-type {asp,slow,history,activity}] [--start-time STAET_TIME] [--end-time END_TIME] [--verify] [--driver] +``` + +The input parameters are as follows: + +- **DB_PORT**: port number of the connected database. +- **DATABASE**: name of the connected database. +- **OUTPUT**: output file including SQLs. +- **DB_HOST** (optional): ID of the host that connects to the database. +- **DB_USER** (optional): username for connecting to the database. The user needs to have the sysadmin or monitor admin permission. +- **SCHEMA**: schema name. It is used only when **statement-type** is set to **history**. The default value is **public**. +- **statement-type**: SQL statement type, including **asp**, **slow**, **history**, and **activity**. + - **asp**: SQL extracted from gs_asp needs to make **enable_asp** enabled. + - **slow**: current active slow SQLs are extracted. + - **history**: historical slow SQLs are extracted. + - **activity**: current active SQLs are extracted. +- **START_TIME**: start time for log collection. It is used only when **statement-type** is set to **asp** and the parameter is mandatory. +- **END_TIME**: end time for log collection. It is used only when **statement-type** is set to **asp** and the parameter is mandatory. +- **verify**: whether to verify the SQL validity. +- **driver**: whether to use a Python driver to connect a database. The default value is **gsql**. ## Procedure for Using the Index Recommendation Script 1. Prepare a file that contains multiple DML statements as the input workload. Each statement in the file occupies a line. You can obtain historical service statements from the offline logs of the database. -2. Run the Python script **index_advisor_workload.py**: +2. To enable this function, run the following command: ``` - python index_advisor_workload.py [p PORT] [d DATABASE] [f FILE] [--h HOST] [-U USERNAME] [-W PASSWORD][--schema SCHEMA][--max_index_num MAX_INDEX_NUM][--max_index_storage MAX_INDEX_STORAGE] [--multi_iter_mode] [--multi_node] [--json] [--driver] [--show_detail] + echo PASSWORD | gs_dbmind component index_advisor [p DB_PORT] [d DATABASE] [f FILE] [--h DB_HOST] [-U DB_USER] [--schema SCHEMA] + [--max_index_num MAX_INDEX_NUM][--max_index_storage MAX_INDEX_STORAGE] [--multi_iter_mode] [--max-n-distinct MAX_N_DISTINCT] + [--min-improved-rate MIN_IMPROVED_RATE] [--max-candidate-columns MAX_CANDIDATE_COLUMNS] [--max-index-columns MAX_INDEX_COLUMNS] + [--min-reltuples MIN_RELTUPLES] [--multi_node] [--json] [--driver] [--show_detail] [--show-benifits] ``` The input parameters are as follows: - - **PORT**: port number of the connected database. + - **DB_PORT**: port number of the connected database. - **DATABASE**: name of the connected database. - **FILE**: file path that contains the workload statement. - - **HOST** (optional): ID of the host that connects to the database. - - **USERNAME** (optional): username for connecting to the database. - - **PASSWORD** (optional): password for connecting to the database. + - **DB_HOST** (optional): ID of the host that connects to the database. + - **DB_USERNAME** (optional): username for connecting to the database. - **SCHEMA**: schema name. - **MAX_INDEX_NUM** (optional): maximum number of recommended indexes. - **MAX_INDEX_STORAGE** (optional): maximum size of the index set space. + - **MAX_N_DISTINCT**: reciprocal value of the number for the distinct value. The default value is **0.01**. + - **MIN_IMPROVED_RATE**: minimum improved rate. The default value is **0.1**. + - **MAX_CANDIDATE_COLUMNS** (optional): maximum number of candidate index columns. + - **MAX_INDEX_COLUMNS**: maximum number of index columns. The default value is **4**. + - **MIN_RELTUPLES**: minimum number of records. The default value is **10000**. - **multi_node** (optional): specifies whether the current instance is a distributed database instance. - **multi_iter_mode** (optional): algorithm mode. You can switch the algorithm mode by setting this parameter. - - **json** (optional): Specifies the file path format of the workload statement as JSON after SQL normalization. By default, each SQL statement occupies one line. - - **driver** (optional): Specifies whether to use the Python driver to connect to the database. By default, **gsql** is used for the connection. - - **show_detail** (optional): Specifies whether to display the detailed optimization information about the current recommended index set. - - Example: - - ``` - python index_advisor_workload.py 6001 postgres tpcc_log.txt --schema public --max_index_num 10 --multi_iter_mode - ``` + - **json** (optional): specifies the file path format of the workload statement as JSON after SQL normalization. By default, each SQL statement occupies one line. + - **driver** (optional): specifies whether to use the Python driver to connect to the database. By default, **gsql** is used for the connection. + - **show_detail** (optional): specifies whether to display the detailed optimization information about the current recommended index set. + - **show-benefits** (optional): whether to show index benefits. The recommendation result is a batch of indexes, which are displayed on the screen in the format of multiple create index statements. The following is an example of the result. - ```sql + ``` create index ind0 on public.bmsql_stock(s_i_id,s_w_id); create index ind1 on public.bmsql_customer(c_w_id,c_id,c_d_id); create index ind2 on public.bmsql_order_line(ol_w_id,ol_o_id,ol_d_id); @@ -100,6 +132,4 @@ For workload-level indexes, you can run scripts outside the database to use this create index ind7 on public.bmsql_new_order(no_w_id); create index ind8 on public.bmsql_oorder(o_w_id,o_c_id,o_d_id); create index ind9 on public.bmsql_district(d_w_id); - ``` - - > ![img](https://cdn-mogdb.enmotech.com/docs-media/icon/icon-note.gif) **NOTE:** The value of the **multi_node** parameter must be specified based on the current database architecture. Otherwise, the recommendation result is incomplete, or even no recommendation result is generated. + ``` \ No newline at end of file diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/3-slow-query-diagnosis-root-cause-analysis-for-slow-sql-statements/3-5-command-reference.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/slow-query-diagnosis-root-cause-analysis-for-slow-sql-statements/slow-sql-statements-command-reference.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/3-slow-query-diagnosis-root-cause-analysis-for-slow-sql-statements/3-5-command-reference.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/slow-query-diagnosis-root-cause-analysis-for-slow-sql-statements/slow-sql-statements-command-reference.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/3-slow-query-diagnosis-root-cause-analysis-for-slow-sql-statements/3-2-environment-deployment.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/slow-query-diagnosis-root-cause-analysis-for-slow-sql-statements/slow-sql-statements-environment-deployment.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/3-slow-query-diagnosis-root-cause-analysis-for-slow-sql-statements/3-2-environment-deployment.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/slow-query-diagnosis-root-cause-analysis-for-slow-sql-statements/slow-sql-statements-environment-deployment.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/3-slow-query-diagnosis-root-cause-analysis-for-slow-sql-statements/3-4-obtaining-help-information.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/slow-query-diagnosis-root-cause-analysis-for-slow-sql-statements/slow-sql-statements-obtaining-help-information.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/3-slow-query-diagnosis-root-cause-analysis-for-slow-sql-statements/3-4-obtaining-help-information.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/slow-query-diagnosis-root-cause-analysis-for-slow-sql-statements/slow-sql-statements-obtaining-help-information.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/3-slow-query-diagnosis-root-cause-analysis-for-slow-sql-statements/3-1-overview.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/slow-query-diagnosis-root-cause-analysis-for-slow-sql-statements/slow-sql-statements-overview.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/3-slow-query-diagnosis-root-cause-analysis-for-slow-sql-statements/3-1-overview.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/slow-query-diagnosis-root-cause-analysis-for-slow-sql-statements/slow-sql-statements-overview.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/3-slow-query-diagnosis-root-cause-analysis-for-slow-sql-statements/3-6-troubleshooting.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/slow-query-diagnosis-root-cause-analysis-for-slow-sql-statements/slow-sql-statements-troubleshooting.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/3-slow-query-diagnosis-root-cause-analysis-for-slow-sql-statements/3-6-troubleshooting.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/slow-query-diagnosis-root-cause-analysis-for-slow-sql-statements/slow-sql-statements-troubleshooting.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/3-slow-query-diagnosis-root-cause-analysis-for-slow-sql-statements/3-3-usage-guide.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/slow-query-diagnosis-root-cause-analysis-for-slow-sql-statements/slow-sql-statements-usage-guide.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/3-slow-query-diagnosis-root-cause-analysis-for-slow-sql-statements/3-3-usage-guide.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/slow-query-diagnosis-root-cause-analysis-for-slow-sql-statements/slow-sql-statements-usage-guide.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/slow-query-diagnosis-root-cause-analysis-for-slow-sql-statements/slow-sql-statements.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/slow-query-diagnosis-root-cause-analysis-for-slow-sql-statements/slow-sql-statements.md new file mode 100644 index 0000000000000000000000000000000000000000..cc4370a880d020778cb986e6332bde3a8a2f5a7c --- /dev/null +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/slow-query-diagnosis-root-cause-analysis-for-slow-sql-statements/slow-sql-statements.md @@ -0,0 +1,15 @@ +--- +title: Slow Query Diagnosis +summary: Slow Query Diagnosis +author: zhang cuiping +date: 2023-04-07 +--- + +# Slow Query Diagnosis: Root Cause Analysis for Slow SQL Statements + +- **[Overview](slow-sql-statements-overview.md)** +- **[Environment Deployment](slow-sql-statements-environment-deployment.md)** +- **[Usage Guide](slow-sql-statements-usage-guide.md)** +- **[Obtaining Help Information](slow-sql-statements-obtaining-help-information.md)** +- **[Command Reference](slow-sql-statements-command-reference.md)** +- **[Troubleshooting](slow-sql-statements-troubleshooting.md)** \ No newline at end of file diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/6-sql-rewriter-sql-statement-rewriting/6-4-command-reference.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/sql-rewriter-sql-statement-rewriting/sql-rewriter-sql-statement-rewriting-command-reference.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/6-sql-rewriter-sql-statement-rewriting/6-4-command-reference.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/sql-rewriter-sql-statement-rewriting/sql-rewriter-sql-statement-rewriting-command-reference.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/6-sql-rewriter-sql-statement-rewriting/6-3-obtaining-help-information.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/sql-rewriter-sql-statement-rewriting/sql-rewriter-sql-statement-rewriting-obtaining-help-information.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/6-sql-rewriter-sql-statement-rewriting/6-3-obtaining-help-information.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/sql-rewriter-sql-statement-rewriting/sql-rewriter-sql-statement-rewriting-obtaining-help-information.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/6-sql-rewriter-sql-statement-rewriting/6-1-overview.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/sql-rewriter-sql-statement-rewriting/sql-rewriter-sql-statement-rewriting-overview.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/6-sql-rewriter-sql-statement-rewriting/6-1-overview.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/sql-rewriter-sql-statement-rewriting/sql-rewriter-sql-statement-rewriting-overview.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/6-sql-rewriter-sql-statement-rewriting/6-5-troubleshooting.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/sql-rewriter-sql-statement-rewriting/sql-rewriter-sql-statement-rewriting-troubleshooting.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/6-sql-rewriter-sql-statement-rewriting/6-5-troubleshooting.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/sql-rewriter-sql-statement-rewriting/sql-rewriter-sql-statement-rewriting-troubleshooting.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/6-sql-rewriter-sql-statement-rewriting/6-2-usage-guide.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/sql-rewriter-sql-statement-rewriting/sql-rewriter-sql-statement-rewriting-usage-guide.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/6-sql-rewriter-sql-statement-rewriting/6-2-usage-guide.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/sql-rewriter-sql-statement-rewriting/sql-rewriter-sql-statement-rewriting-usage-guide.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/sql-rewriter-sql-statement-rewriting/sql-rewriter-sql-statement-rewriting.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/sql-rewriter-sql-statement-rewriting/sql-rewriter-sql-statement-rewriting.md new file mode 100644 index 0000000000000000000000000000000000000000..7fa8793422241317dba53f4f402dc6276813f909 --- /dev/null +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/sql-rewriter-sql-statement-rewriting/sql-rewriter-sql-statement-rewriting.md @@ -0,0 +1,14 @@ +--- +title: SQL Rewriter +summary: SQL Rewriter +author: zhang cuiping +date: 2023-04-07 +--- + +# SQL Rewriter: SQL Statement Rewriting + +- **[Overview](sql-rewriter-sql-statement-rewriting-overview.md)** +- **[Usage Guide](sql-rewriter-sql-statement-rewriting-usage-guide.md)** +- **[Obtaining Help Information](sql-rewriter-sql-statement-rewriting-obtaining-help-information.md)** +- **[Command Reference](sql-rewriter-sql-statement-rewriting-command-reference.md)** +- **[Troubleshooting](sql-rewriter-sql-statement-rewriting-troubleshooting.md)** \ No newline at end of file diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/5-sqldiag-slow-sql-discovery/5-4-command-reference.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/sqldiag-slow-sql-discovery/sqldiag-slow-sql-discovery-command-reference.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/5-sqldiag-slow-sql-discovery/5-4-command-reference.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/sqldiag-slow-sql-discovery/sqldiag-slow-sql-discovery-command-reference.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/5-sqldiag-slow-sql-discovery/5-3-obtaining-help-information.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/sqldiag-slow-sql-discovery/sqldiag-slow-sql-discovery-obtaining-help-information.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/5-sqldiag-slow-sql-discovery/5-3-obtaining-help-information.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/sqldiag-slow-sql-discovery/sqldiag-slow-sql-discovery-obtaining-help-information.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/5-sqldiag-slow-sql-discovery/5-1-overview.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/sqldiag-slow-sql-discovery/sqldiag-slow-sql-discovery-overview.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/5-sqldiag-slow-sql-discovery/5-1-overview.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/sqldiag-slow-sql-discovery/sqldiag-slow-sql-discovery-overview.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/5-sqldiag-slow-sql-discovery/5-5-troubleshooting.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/sqldiag-slow-sql-discovery/sqldiag-slow-sql-discovery-troubleshooting.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/5-sqldiag-slow-sql-discovery/5-5-troubleshooting.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/sqldiag-slow-sql-discovery/sqldiag-slow-sql-discovery-troubleshooting.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/5-sqldiag-slow-sql-discovery/5-2-usage-guide.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/sqldiag-slow-sql-discovery/sqldiag-slow-sql-discovery-usage-guide.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/5-sqldiag-slow-sql-discovery/5-2-usage-guide.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/sqldiag-slow-sql-discovery/sqldiag-slow-sql-discovery-usage-guide.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/sqldiag-slow-sql-discovery/sqldiag-slow-sql-discovery.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/sqldiag-slow-sql-discovery/sqldiag-slow-sql-discovery.md new file mode 100644 index 0000000000000000000000000000000000000000..5178d7074d52e80d6582f48f1e4b6c497d5ad869 --- /dev/null +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/sqldiag-slow-sql-discovery/sqldiag-slow-sql-discovery.md @@ -0,0 +1,22 @@ +--- +title: SQLdiag +summary: SQLdiag +author: zhang cuiping +date: 2023-04-07 +--- + +# SQLdiag: Slow SQL Discovery + +SQLdiag is a framework for predicting the execution duration of SQL statements in openGauss. The existing prediction technologies are mainly based on model prediction of execution plans. These prediction solutions are applicable only to jobs whose execution plans can be obtained in the OLAP scenarios, and are not useful for quick query such as OLTP or HTAP. Different from the preceding solutions, SQLdiag focuses on the historical SQL statements of databases. Because the execution duration of the database SQL statements in a short time does not vary greatly, SQLdiag can detect instruction sets similar to the entered instructions from the historical data, and predict the SQL statement execution duration based on the SQL vectorization technology and the time series prediction algorithm. This framework has the following benefits: + +1. Execution plans do not require instructions. This has no impact on database performance. +2. The framework is widely used, unlike many other well-targeted algorithms in the industry, for example, they may applicable only to OLTP or OLAP. +3. The framework is robust and easy to understand. Users can design their own prediction models by simply modifying the framework. + +The typical application scenario of this tool is to detect a batch of SQL statements to be online so as to identify risks in advance. + +- **[Overview](sqldiag-slow-sql-discovery-overview.md)** +- **[Usage Guide](sqldiag-slow-sql-discovery-usage-guide.md)** +- **[Obtaining Help Information](sqldiag-slow-sql-discovery-obtaining-help-information.md)** +- **[Command Reference](sqldiag-slow-sql-discovery-command-reference.md)** +- **[Troubleshooting](sqldiag-slow-sql-discovery-troubleshooting.md)** \ No newline at end of file diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/1-x-tuner-parameter-optimization-and-diagnosis/1-5-command-reference.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/x-tuner-parameter-optimization-and-diagnosis/x-tuner-command-reference.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/1-x-tuner-parameter-optimization-and-diagnosis/1-5-command-reference.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/x-tuner-parameter-optimization-and-diagnosis/x-tuner-command-reference.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/1-x-tuner-parameter-optimization-and-diagnosis/1-3-examples.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/x-tuner-parameter-optimization-and-diagnosis/x-tuner-examples.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/1-x-tuner-parameter-optimization-and-diagnosis/1-3-examples.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/x-tuner-parameter-optimization-and-diagnosis/x-tuner-examples.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/x-tuner-parameter-optimization-and-diagnosis/x-tuner-obtaining-help-information.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/x-tuner-parameter-optimization-and-diagnosis/x-tuner-obtaining-help-information.md new file mode 100644 index 0000000000000000000000000000000000000000..9536cea7e3261b6e7eadf8e75d78e87586904b29 --- /dev/null +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/x-tuner-parameter-optimization-and-diagnosis/x-tuner-obtaining-help-information.md @@ -0,0 +1,48 @@ +--- +title: Obtaining Help Information +summary: Obtaining Help Information +author: Guo Huan +date: 2021-05-19 +--- + +# Obtaining Help Information + +Before starting the tuning program, run the following command to obtain help information: + +```bash +gs_dbmind component xtuner --help +``` + +The command output is as follows: + +```bash +usage: [-h] [--database DATABASE] [--db-user DB_USER] [--db-port DB_PORT] [--db-host DB_HOST] [--host-user HOST_USER] [--host-ssh-port HOST_SSH_PORT] [-f DB_CONFIG_FILE] [-x TUNER_CONFIG_FILE] [-v] + {train,tune,recommend} + +X-Tuner: a self-tuning tool integrated by openGauss. + +positional arguments: + {train,tune,recommend} + Train a reinforcement learning model or tune database by model. And also can recommend best_knobs according to your workload. + +optional arguments: + -h, --help show this help message and exit + -f DB_CONFIG_FILE, --db-config-file DB_CONFIG_FILE + You can pass a path of configuration file otherwise you should enter database information by command arguments manually. Please see the template file share/server.json.template. + -x TUNER_CONFIG_FILE, --tuner-config-file TUNER_CONFIG_FILE + This is the path of the core configuration file of the X-Tuner. You can specify the path of the new configuration file. The default path is /path/to/config/file. You can modify the configuration file to control the tuning process. + -v, --version show program's version number and exit + +Database Connection Information: + --database DATABASE, --db-name DATABASE + The name of database where your workload running on. + --db-user DB_USER Use this user to login your database. Note that the user must have sufficient permissions. + --db-port DB_PORT, --port DB_PORT + Use this port to connect with the database. + --db-host DB_HOST, --host DB_HOST + The IP address of your database InstallationGuide host. + --host-user HOST_USER + The login user of your database InstallationGuide host. + --host-ssh-port HOST_SSH_PORT + The SSH port of your database InstallationGuide host. +``` diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/1-x-tuner-parameter-optimization-and-diagnosis/1-1-x-tuner-overview.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/x-tuner-parameter-optimization-and-diagnosis/x-tuner-overview.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/1-x-tuner-parameter-optimization-and-diagnosis/1-1-x-tuner-overview.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/x-tuner-parameter-optimization-and-diagnosis/x-tuner-overview.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/x-tuner-parameter-optimization-and-diagnosis/x-tuner-parameter-optimization-and-diagnosis.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/x-tuner-parameter-optimization-and-diagnosis/x-tuner-parameter-optimization-and-diagnosis.md new file mode 100644 index 0000000000000000000000000000000000000000..0d9ec2363bfb7b08fb60e1b7d0dd0a4a1e483e36 --- /dev/null +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/x-tuner-parameter-optimization-and-diagnosis/x-tuner-parameter-optimization-and-diagnosis.md @@ -0,0 +1,15 @@ +--- +title: X-Tuner +summary: X-Tuner +author: zhang cuiping +date: 2023-04-07 +--- + +# X-Tuner: Parameter Tuning and Diagnosis + +- **[Overview](x-tuner-overview.md)** +- **[Preparations](x-tuner-preparations.md)** +- **[Examples](x-tuner-examples.md)** +- **[Obtaining Help Information](x-tuner-obtaining-help-information.md)** +- **[Command Reference](x-tuner-command-reference.md)** +- **[Troubleshooting](x-tuner-troubleshooting.md)** \ No newline at end of file diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/1-x-tuner-parameter-optimization-and-diagnosis/1-2-preparations.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/x-tuner-parameter-optimization-and-diagnosis/x-tuner-preparations.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/1-x-tuner-parameter-optimization-and-diagnosis/1-2-preparations.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/x-tuner-parameter-optimization-and-diagnosis/x-tuner-preparations.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/1-x-tuner-parameter-optimization-and-diagnosis/1-6-Troubleshooting.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/x-tuner-parameter-optimization-and-diagnosis/x-tuner-troubleshooting.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/1-x-tuner-parameter-optimization-and-diagnosis/1-6-Troubleshooting.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/x-tuner-parameter-optimization-and-diagnosis/x-tuner-troubleshooting.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/components-that-support-dbmind/components-that-support-dbmind.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/components-that-support-dbmind/components-that-support-dbmind.md index 88b4b69ef04099d24c9f334f82b7b2670ca5b588..767a9abbe62a88dc07cccc0fdfb9d447114288e7 100644 --- a/product/en/docs-mogdb/v5.0/AI-features/ai4db/components-that-support-dbmind/components-that-support-dbmind.md +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/components-that-support-dbmind/components-that-support-dbmind.md @@ -8,3 +8,5 @@ date: 2022-05-06 # Components that Support DBMind DBMind provides components to support the deployment and implementation of the entire service or solution. They are not AI functions, but are an important part of the entire service system. They are used to support the quick implementation of the entire autonomous O&M solution. For example, the exporter is used to collect database metrics. + +- **[Prometheus Exporter](./prometheus-exporter/prometheus-exporter.md)** \ No newline at end of file diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/components-that-support-dbmind/prometheus-exporter/prometheus-exporter.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/components-that-support-dbmind/prometheus-exporter/prometheus-exporter.md new file mode 100644 index 0000000000000000000000000000000000000000..b8d0096a9286c01c22311773360d1accfea652de --- /dev/null +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/components-that-support-dbmind/prometheus-exporter/prometheus-exporter.md @@ -0,0 +1,15 @@ +--- +title: Prometheus Exporter +summary: Prometheus Exporter +author: zhang cuiping +date: 2023-04-07 +--- + +# Prometheus Exporter + +- **[Overview](prometheus-exporter-overview.md)** +- **[Environment Deployment](prometheus-exporter-environment-deployment.md)** +- **[Usage Guide](prometheus-exporter-usage-guide.md)** +- **[Obtaining Help Information](prometheus-exporter-obtaining-help-information.md)** +- **[Command Reference](prometheus-exporter-command-reference.md)** +- **[Troubleshooting](prometheus-exporter-troubleshooting.md)** \ No newline at end of file diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/dbmind-mode/2-component.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/dbmind-mode/component.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/dbmind-mode/2-component.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/dbmind-mode/component.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/dbmind-mode/dbmind-mode.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/dbmind-mode/dbmind-mode.md index 108623f4121e22e61a56ec13fdc92927b1bc6233..a60f1a292bbf3c95a7fde3cc70f01c82b0ba981d 100644 --- a/product/en/docs-mogdb/v5.0/AI-features/ai4db/dbmind-mode/dbmind-mode.md +++ b/product/en/docs-mogdb/v5.0/AI-features/ai4db/dbmind-mode/dbmind-mode.md @@ -44,4 +44,8 @@ available subcommands: | --version | Version number | - | | service | Subcommand related to service functions | - | | component | Subcommand for invoking components | - | -| set | Subcommand for modifying the configuration file | - | \ No newline at end of file +| set | Subcommand for modifying the configuration file | - | + +- **[Service](service.md)** +- **[Component](component.md)** +- **[Set](set.md)** \ No newline at end of file diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/dbmind-mode/1-service.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/dbmind-mode/service.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/dbmind-mode/1-service.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/dbmind-mode/service.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/ai4db/dbmind-mode/3-set.md b/product/en/docs-mogdb/v5.0/AI-features/ai4db/dbmind-mode/set.md similarity index 100% rename from product/en/docs-mogdb/v5.0/AI-features/ai4db/dbmind-mode/3-set.md rename to product/en/docs-mogdb/v5.0/AI-features/ai4db/dbmind-mode/set.md diff --git a/product/en/docs-mogdb/v5.0/AI-features/db4ai/db4ai.md b/product/en/docs-mogdb/v5.0/AI-features/db4ai/db4ai.md index 7ea0e6e50a2a6fbf8b1ae511e6bfca012b165886..4bf0faca9ee7cd09620e0540e669a8cb4e1d9349 100644 --- a/product/en/docs-mogdb/v5.0/AI-features/db4ai/db4ai.md +++ b/product/en/docs-mogdb/v5.0/AI-features/db4ai/db4ai.md @@ -8,3 +8,6 @@ date: 2022-05-06 # DB4AI Database-driven AI DB4AI uses database capabilities to drive AI tasks and implement data storage and technology stack isomorphism. By integrating AI algorithms into the database, MogDB supports the native AI computing engine, model management, AI operators, and native AI execution plan, providing users with inclusive AI technologies. Different from the traditional AI modeling process, DB4AI one-stop modeling eliminates repeated data flowing among different platforms, simplifies the development process, and plans the optimal execution path through the database, so that developers can focus on the tuning of specific services and models. It outcompetes similar products in ease-of-use and performance. + +- **[Native DB4AI Engine](native-db4ai-engine.md)** +- **[Full-process AI](./full-process-ai/full-process-ai.md)** \ No newline at end of file diff --git a/product/en/docs-mogdb/v5.0/AI-features/db4ai/full-process-ai/db4ai-query-for-model-training-and-prediction.md b/product/en/docs-mogdb/v5.0/AI-features/db4ai/full-process-ai/db4ai-query-for-model-training-and-prediction.md new file mode 100644 index 0000000000000000000000000000000000000000..bbddf60f1472aaa9c43146d523f51ba312d27ba9 --- /dev/null +++ b/product/en/docs-mogdb/v5.0/AI-features/db4ai/full-process-ai/db4ai-query-for-model-training-and-prediction.md @@ -0,0 +1,310 @@ +--- +title: DB4AI-Query +summary: DB4AI-Query +author: zhang cuiping +date: 2023-04-07 +--- + +# DB4AI-Query for Model Training and Prediction + +The current version of MogDB supports the native DB4AI capability. By introducing native AI operators, MogDB simplifies the operation process and fully utilizes the optimization and execution capabilities of the database optimizer and executor to obtain the high-performance model training capability in the database. With a simpler model training and prediction process and higher performance, developers can focus on model tuning and data analysis in a shorter period of time, avoiding fragmented technology stacks and redundant code implementation. + +## Keyword Parsing + +**Table 1** DB4AI syntax and keywords + +| Name | Description | +| ------------ | ---------------------------------------------------------- | +| CREATE MODEL | Creates a model, trains it, and saves the model. | +| PREDICT BY | Uses an existing model for prediction. | +| TARGET | Target column name of a training or prediction task. | +| FEATURES | Data feature column name of a training or prediction task. | +| MODEL | Model name of a training task. | + +## Developer Guide + +1. Introduce the algorithms supported in this version. + + DB4AI of the current version supports logistic regression (binary classification tasks), linear regression, and vector machine algorithms (classification tasks) based on the SGD operator, as well as the K-Means clustering algorithm based on the K-Means operator. + +2. Learn about the model training syntax. + + - CREATE MODEL + + You can run the **CREATE MODEL** statement to create and train a model. Taking dataset **kmeans_2d** as an example, the data content of the table is as follows: + + ``` + MogDB=# select * from kmeans_2d; + id | position + ----+------------------------------------- + 1 | {74.5268815685995,88.2141939294524} + 2 | {70.9565760521218,98.8114827475511} + 3 | {76.2756086327136,23.8387574302033} + 4 | {17.8495847294107,81.8449544720352} + 5 | {81.2175785354339,57.1677675866522} + 6 | {53.97752255667,49.3158342130482} + 7 | {93.2475341879763,86.934042100329} + 8 | {72.7659293473698,19.7020415100269} + 9 | {16.5800288529135,75.7475957670249} + 10 | {81.8520747194998,40.3476078575477} + 11 | {76.796671198681,86.3827232690528} + 12 | {59.9231450678781,90.9907738864422} + 13 | {70.161884885747,19.7427458665334} + 14 | {11.1269539105706,70.9988166182302} + 15 | {80.5005071521737,65.2822235273197} + 16 | {54.7030725912191,52.151339428965} + 17 | {103.059707058128,80.8419883321039} + 18 | {85.3574452036992,14.9910179991275} + 19 | {28.6501615960151,76.6922890325077} + 20 | {69.7285806713626,49.5416352967732} + (20 rows) + ``` + + The data type of the **position** field in this table is double precision[]. + + - The following uses K-Means as an example to describe how to train a model. Specify **position** as a feature column in the **kmeans_2d** training set, and use the K-Means algorithm to create and save the **point_kmeans** model. + + ``` + MogDB=# CREATE MODEL point_kmeans USING kmeans FEATURES position FROM kmeans_2d WITH num_centroids=3; + NOTICE: Hyperparameter max_iterations takes value DEFAULT (10) + NOTICE: Hyperparameter num_centroids takes value 3 + NOTICE: Hyperparameter tolerance takes value DEFAULT (0.000010) + NOTICE: Hyperparameter batch_size takes value DEFAULT (10) + NOTICE: Hyperparameter num_features takes value DEFAULT (2) + NOTICE: Hyperparameter distance_function takes value DEFAULT (L2_Squared) + NOTICE: Hyperparameter seeding_function takes value DEFAULT (Random++) + NOTICE: Hyperparameter verbose takes value DEFAULT (0) + NOTICE: Hyperparameter seed takes value DEFAULT (0) + MODEL CREATED. PROCESSED 1 + ``` + + In the preceding command: + + - The **CREATE MODEL** statement is used to train and save a model. + + - **USING** specifies the algorithm name. + + - **FEATURES** specifies the features of the training model and needs to be added based on the column name of the training data table. + + - **TARGET** specifies the training target of the model. It can be the column name of the data table required for training or an expression, for example, **price > 10000**. + + - **WITH** specifies the hyperparameters used for model training. When the hyperparameter is not set by the user, the framework uses the default value. + + The framework supports various hyperparameter combinations for different operators. + + **Table 2** Hyperparameters supported by operators + + | Operator | Hyperparameter | + | :----------------------------------------------------------- | :----------------------------------------------------------- | + | GD(logistic_regression, linear_regression, and svm_classification) | optimizer(char*), verbose(bool), max_iterations(int), max_seconds(double), batch_size(int), learning_rate(double), decay(double), and tolerance(double)SVM limits the hyperparameter **lambda(double)**. | + | K-Means | max_iterations(int), num_centroids(int), tolerance(double), batch_size(int), num_features(int), distance_function(char*), seeding_function(char*), verbose(int), and seed(int) | + + The default value and value range of each hyperparameter are as follows: + + **Table 3** Default values and value ranges of hyperparameters + + | Operator | Default Hyperparameter Value | Value Range | Hyperparameter Description | + | :----------------------------------------------------------- | :------------------------------------------ | :------------------------------------------- | :------------------------- | + | GD (logistic_regression, linear_regression, and svm_classification) | optimizer = gd (gradient descent) | **gd** or **ngd** (natural gradient descent) | Optimizer | + | verbose = false | **T** or **F** | Log display | | + | max_iterations = 100 | (0,INT_MAX_VALUE] | Maximum iterations | | + | max_seconds = 0 (The running duration is not limited.) | [0,INT_MAX_VALUE] | Running duration | | + | batch_size = 1000 | (0,MAX_MEMORY_LIMIT] | Number of data records selected per training | | + | learning_rate = 0.8 | (0,DOUBLE_MAX_VALUE] | Learning rate | | + | decay = 0.95 | (0,DOUBLE_MAX_VALUE] | Weight decay rate | | + | tolerance = 0.0005 | (0,DOUBLE_MAX_VALUE] | Tolerance | | + | seed = 0 (random value of **seed**) | [0,INT_MAX_VALUE] | Seed | | + | just for SVM:lambda = 0.01 | (0,DOUBLE_MAX_VALUE) | Regularization parameter | | + | K-Means | max_iterations = 10 | [1,INT_MAX_VALUE] | Maximum iterations | + | num_centroids = 10 | [1,MAX_MEMORY_LIMIT] | Number of clusters | | + | tolerance = 0.00001 | (0,1) | Central point error | | + | batch_size = 10 | [1,MAX_MEMORY_LIMIT] | Number of data records selected per training | | + | num_features = 2 | [1,GS_MAX_COLS] | Number of sample features | | + | distance_function = "L2_Squared" | **L1**, **L2**, **L2_Squared**, or **Linf** | Regularization method | | + | seeding_function = "Random++" | **"Random++"** or **"KMeans\|\|"** | Method for initializing seed points | | + | verbose = 0U | {0,1,2} | Verbose mode | | + | seed = 0U | [0,INT_MAX_VALUE] | Seed | | + | MAX_MEMORY_LIMIT = Maximum number of tuples loaded in memory | | | | + | GS_MAX_COLS = Maximum number of attributes in a database table | | | | + + - If the model is saved successfully, the following information is returned: + + ``` + MODEL CREATED. PROCESSED x + ``` + +3. View the model information. + + After the training is complete, the model is stored in the **gs_model_warehouse** system catalog. You can view information about the model and training process in the **gs_model_warehouse** system catalog. + + You can view a model by viewing the system catalog. For example, run the following SQL statement to view the model named **point_kmeans**: + + ``` + MogDB=# select * from gs_model_warehouse where modelname='point_kmeans'; + -[ RECORD 1 ]---------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + modelname | point_kmeans + modelowner | 10 + createtime | 2021-04-30 17:30:39.59044 + processedtuples | 20 + discardedtuples | 0 + pre_process_time | 6.2001e-05 + exec_time | .000185272 + iterations | 5 + outputtype | 23 + modeltype | kmeans + query | CREATE MODEL point_kmeans USING kmeans FEATURES position FROM kmeans_2d WITH num_centroids=3; + modeldata | + weight | + hyperparametersnames | {max_iterations,num_centroids,tolerance,batch_size,num_features,distance_function,seeding_function,verbose,seed} + hyperparametersvalues | {10,3,1e-05,10,2,L2_Squared,Random++,0,0} + hyperparametersoids | {23,23,701,23,23,1043,1043,23,23} + coefnames | {original_num_centroids,actual_num_centroids,dimension,distance_function_id,seed,coordinates} + coefvalues | {3,3,2,2,572368998,"(77.282589,23.724434)(74.421616,73.239455)(18.551682,76.320914)"} + coefoids | + trainingscoresname | + trainingscoresvalue | + modeldescribe | {"id:1,objective_function:542.851169,avg_distance_to_centroid:108.570234,min_distance_to_centroid:1.027078,max_distance_to_centroid:297.210108,std_dev_distance_to_centroid:105.053257,cluster_size:5","id:2,objective_function:5825.982139,avg_distance_to_centroid:529.634740,min_distance_to_centroid:100.270449,max_distance_to_centroid:990.300588,std_dev_distance_to_centroid:285.915094,cluster_size:11","id:3,objective_function:220.792591,avg_distance_to_centroid:55.198148,min_distance_to_centroid:4.216111,max_distance_to_centroid:102.117204,std_dev_distance_to_centroid:39.319118,cluster_size:4"} + ``` + +4. Use an existing model to perform a prediction task. + + Use the **SELECT** and **PREDICT BY** keywords to complete the prediction task based on the existing model. + + Query syntax: SELECT… PREDICT BY… (FEATURES…)… FROM…; + + ``` + MogDB=# SELECT id, PREDICT BY point_kmeans (FEATURES position) as pos FROM (select * from kmeans_2d limit 10); + id | pos + ----+----- + 1 | 2 + 2 | 2 + 3 | 1 + 4 | 3 + 5 | 2 + 6 | 2 + 7 | 2 + 8 | 1 + 9 | 3 + 10 | 1 + (10 rows) + ``` + + For the same prediction task, the results of the same model are stable. In addition, models trained based on the same hyperparameter and training set are stable. AI model training is random (random gradient descent of data distribution each batch). Therefore, the computing performance and results of different models can vary slightly. + +5. View the execution plan. + + You can use the **EXPLAIN** statement to analyze the execution plan in the model training or prediction process of **CREATE MODEL** and **PREDICT BY**. The keyword **EXPLAIN** can be followed by a **CREATE MODEL** or **PREDICT BY** clause or an optional parameter. The supported parameters are as follows: + + **Table 4** Parameters supported by EXPLAIN + + | Parameter | Description | + | :-------- | :----------------------------------------------------------- | + | ANALYZE | Boolean variable, which is used to add description information such as the running time and number of loop times | + | VERBOSE | Boolean variable, which determines whether to output the training running information to the client | + | COSTS | Boolean variable | + | CPU | Boolean variable | + | DETAIL | Boolean variable, which is available only in distributed mode | + | NODES | Boolean variable, which is available only in distributed mode | + | NUM_NODES | Boolean variable, which is available only in distributed mode | + | BUFFERS | Boolean variable | + | TIMING | Boolean variable | + | PLAN | Boolean variable | + | FORMAT | Optional format type: TEXT, XML, JSON, and YAML | + + Example: + + ``` + MogDB=# Explain CREATE MODEL patient_logisitic_regression USING logistic_regression FEATURES second_attack, treatment TARGET trait_anxiety > 50 FROM patients WITH batch_size=10, learning_rate = 0.05; + NOTICE: Hyperparameter batch_size takes value 10 + NOTICE: Hyperparameter decay takes value DEFAULT (0.950000) + NOTICE: Hyperparameter learning_rate takes value 0.050000 + NOTICE: Hyperparameter max_iterations takes value DEFAULT (100) + NOTICE: Hyperparameter max_seconds takes value DEFAULT (0) + NOTICE: Hyperparameter optimizer takes value DEFAULT (gd) + NOTICE: Hyperparameter tolerance takes value DEFAULT (0.000500) + NOTICE: Hyperparameter seed takes value DEFAULT (0) + NOTICE: Hyperparameter verbose takes value DEFAULT (FALSE) + NOTICE: GD shuffle cache size 212369 + QUERY PLAN + ------------------------------------------------------------------- + Gradient Descent (cost=0.00..0.00 rows=0 width=0) + -> Seq Scan on patients (cost=0.00..32.20 rows=1776 width=12) + (2 rows) + ``` + +6. Perform troubleshooting in case of exceptions. + + - Training phase + + - Scenario 1: When the value of the hyperparameter exceeds the value range, the model training fails and an error message is returned. For example: + + ``` + MogDB=# CREATE MODEL patient_linear_regression USING linear_regression FEATURES second_attack,treatment TARGET trait_anxiety FROM patients WITH optimizer='aa'; + NOTICE: Hyperparameter batch_size takes value DEFAULT (1000) + NOTICE: Hyperparameter decay takes value DEFAULT (0.950000) + NOTICE: Hyperparameter learning_rate takes value DEFAULT (0.800000) + NOTICE: Hyperparameter max_iterations takes value DEFAULT (100) + NOTICE: Hyperparameter max_seconds takes value DEFAULT (0) + NOTICE: Hyperparameter optimizer takes value aa + ERROR: Invalid hyperparameter value for optimizer. Valid values are: gd, ngd. (default is gd) + ``` + + - Scenario 2: If the model name already exists, the model fails to be saved, and an error message with the cause is displayed: + + ``` + MogDB=# CREATE MODEL patient_linear_regression USING linear_regression FEATURES second_attack,treatment TARGET trait_anxiety FROM patients; + NOTICE: Hyperparameter batch_size takes value DEFAULT (1000) + NOTICE: Hyperparameter decay takes value DEFAULT (0.950000) + NOTICE: Hyperparameter learning_rate takes value DEFAULT (0.800000) + NOTICE: Hyperparameter max_iterations takes value DEFAULT (100) + NOTICE: Hyperparameter max_seconds takes value DEFAULT (0) + NOTICE: Hyperparameter optimizer takes value DEFAULT (gd) + NOTICE: Hyperparameter tolerance takes value DEFAULT (0.000500) + NOTICE: Hyperparameter seed takes value DEFAULT (0) + NOTICE: Hyperparameter verbose takes value DEFAULT (FALSE) + NOTICE: GD shuffle cache size 5502 + ERROR: The model name "patient_linear_regression" already exists in gs_model_warehouse. + ``` + + - Scenario 3: If the value in the **FEATURE** or **TARGETS** column is *****, **ERROR** is returned with the error cause: + + ``` + MogDB=# CREATE MODEL patient_linear_regression USING linear_regression FEATURES * TARGET trait_anxiety FROM + patients; + ERROR: FEATURES clause cannot be * + ----------------------------------------------------------------------------------------------------------------------- + MogDB=# CREATE MODEL patient_linear_regression USING linear_regression FEATURES second_attack,treatment TARGET * FROM patients; + ERROR: TARGET clause cannot be * + ``` + + - Scenario 4: If the keyword **TARGET** is used in the unsupervised learning method or is not applicable to the supervised learning method, **ERROR** is returned with the error cause: + + ``` + MogDB=# CREATE MODEL patient_linear_regression USING linear_regression FEATURES second_attack,treatment FROM patients; + ERROR: Supervised ML algorithms require TARGET clause + ----------------------------------------------------------------------------------------------------------------------------- + CREATE MODEL patient_linear_regression USING linear_regression TARGET trait_anxiety FROM patients; ERROR: Supervised ML algorithms require FEATURES clause + ``` + + - Scenario 5: If the GUC parameter **statement_timeout** is set, the statement that is executed due to training timeout will be terminated. In this case, execute the **CREATE MODEL** statement. Parameters such as the size of the training set, number of training rounds (**iteration**), early termination conditions (**tolerance** and **max_seconds**), and number of parallel threads (**nthread**) affect the training duration. When the duration exceeds the database limit, the statement execution is terminated and model training fails. + + - Prediction phase + + - Scenario 6: If the model name cannot be found in the system catalog, the database reports **ERROR**: + + ``` + MogDB=# select id, PREDICT BY patient_logistic_regression (FEATURES second_attack,treatment) FROM patients; + ERROR: There is no model called "patient_logistic_regression". + ``` + + - Scenario 7: If the data dimension and data type of the **FEATURES** task are inconsistent with those of the training set, **ERROR** is reported and the error cause is displayed. For example: + + ``` + MogDB=# select id, PREDICT BY patient_linear_regression (FEATURES second_attack) FROM patients; + ERROR: Invalid number of features for prediction, provided 1, expected 2 + CONTEXT: referenced column: patient_linear_regression_pred + ------------------------------------------------------------------------------------------------------------------------------------- + MogDB=# select id, PREDICT BY patient_linear_regression (FEATURES 1,second_attack,treatment) FROM patients; + ERROR: Invalid number of features for prediction, provided 3, expected 2 + CONTEXT: referenced column: patient_linear_regression_pre + ``` \ No newline at end of file diff --git a/product/en/docs-mogdb/v5.0/AI-features/db4ai/full-process-ai/full-process-ai.md b/product/en/docs-mogdb/v5.0/AI-features/db4ai/full-process-ai/full-process-ai.md index df6e65012d8f72fb7c422580f10c8e9913e95839..89167026434d8fff86717adcb1b0250d9aa313ee 100644 --- a/product/en/docs-mogdb/v5.0/AI-features/db4ai/full-process-ai/full-process-ai.md +++ b/product/en/docs-mogdb/v5.0/AI-features/db4ai/full-process-ai/full-process-ai.md @@ -10,3 +10,7 @@ date: 2022-05-06 A traditional AI task usually has multiple processes. For example, a data collection process includes data collection, data cleaning, and data storage, an algorithm training process includes data preprocessing, training, and model storage and management, and a model training process includes hyperparameter tuning. The entire lifecycle of such machine learning models can be integrated into the database. The model training, management, and optimization processes are performed closest to the data storage. The out-of-the-box AI lifecycle management function of SQL statements is provided on the database side, which is called full-process AI. MogDB implements some full-process AI functions, which will be described in detail in this section. + +- **[PLPython Fenced Mode](plpython-fenced-mode.md)** +- **[DB4AI-Snapshots for Data Version Management](db4ai-snapshots-for-data-version-management.md)** +- **[DB4AI-Query for Model Training and Prediction](db4ai-query-for-model-training-and-prediction.md)** \ No newline at end of file diff --git a/product/zh/docs-mogdb/v5.0/AI-features/ai-feature.md b/product/zh/docs-mogdb/v5.0/AI-features/ai-feature.md index e389285bf9f539aef4a7d9bfe21ba59524388399..29ef473b804b5231c7aa911d12566e06f03d8f18 100644 --- a/product/zh/docs-mogdb/v5.0/AI-features/ai-feature.md +++ b/product/zh/docs-mogdb/v5.0/AI-features/ai-feature.md @@ -9,7 +9,7 @@ date: 2022-05-06 人工智能技术最早可以追溯到上世纪50年代，甚至比数据库系统的发展历史还要悠久。但是，由于各种各样客观因素的制约，在很长的一段时间内，人工智能技术并没有得到大规模的应用，甚至还经历了几次明显的低谷期。到了近些年，随着信息技术的进一步发展，从前限制人工智能发展的因素已经逐渐减弱，所谓的ABC（AI、Big data、Cloud computing）技术也随之而诞生。 -AI与数据库结合是近些年的行业研究热点，MogDB较早地参与了该领域的探索，并取得了阶段性的成果。AI特性子模块名为DBMind，相对数据库其他功能更为独立，大致可分为AI4DB、DB4AI和ABO优化器三个部分。 +AI与数据库结合是近些年的行业研究热点，MogDB较早地参与了该领域的探索，并取得了阶段性的成果。AI特性子模块名为DBMind，相对数据库其他功能更为独立，大致可分为AI4DB和DB4AI两个部分。 - AI4DB就是指用人工智能技术优化数据库的性能，从而获得更好地执行表现；也可以通过人工智能的手段实现自治、免运维等。主要包括自调优、自诊断、自安全、自运维、自愈等子领域； - DB4AI就是指打通数据库到人工智能应用的端到端流程，通过数据库来驱动AI任务，统一人工智能技术栈，达到开箱即用、高性能、节约成本等目的。例如通过SQL-like语句实现推荐系统、图像检索、时序预测等功能，充分发挥数据库的高并行、列存储等优势，既可以避免数据和碎片化存储的代价，又可以避免因信息泄漏造成的安全风险； diff --git a/product/zh/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-analysis/anomaly-analysis-multi-metric-correlation-analysis.md b/product/zh/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-analysis/anomaly-analysis-multi-metric-correlation-analysis.md deleted file mode 100644 index 77f86a0c95e72532c7fdc452414453025a21c13d..0000000000000000000000000000000000000000 --- a/product/zh/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/anomaly-analysis/anomaly-analysis-multi-metric-correlation-analysis.md +++ /dev/null @@ -1,18 +0,0 @@ ---- -title: 多指标关联分析 -summary: 多指标关联分析 -author: zhang cuiping -date: 2023-04-07 ---- - -# Anomaly-analysis-多指标关联分析 - -- **[概述](anomaly-analysis-overview.md)** - -- **[使用指导](anomaly-analysis-usage-guide.md)** - -- **[获取帮助](anomaly-analysis-obtaining-help-information.md)** - -- **[命令参考](anomaly-analysis-command-reference.md)** - -- **[常见问题处理](anomaly-analysis-troubleshooting.md)** \ No newline at end of file diff --git a/product/zh/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/index-advisor-index-recommendation/workload-level-index-recommendation.md b/product/zh/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/index-advisor-index-recommendation/workload-level-index-recommendation.md index 021d600b317dd05f7c9b953ec906c6d4d7b85a5b..0ef7f37a6160a5e4c97abbbd187bb79e5f74a8b6 100644 --- a/product/zh/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/index-advisor-index-recommendation/workload-level-index-recommendation.md +++ b/product/zh/docs-mogdb/v5.0/AI-features/ai4db/ai-sub-functions-of-the-dbmind/index-advisor-index-recommendation/workload-level-index-recommendation.md @@ -25,7 +25,7 @@ date: 2021-05-19 2. 执行根据日志抽取SQL语句的功能，命令如下： ``` - ""gs_dbmind component extract_log [l LOG_DIRECTORY] [f OUTPUT_FILE] [p LOG_LINE_PREFIX] [-d DATABASE] [-U USERNAME][--start_time] [--sql_amount] [--statement] [--json] [--max_reserved_period] [--max_template_num] + gs_dbmind component extract_log [l LOG_DIRECTORY] [f OUTPUT_FILE] [p LOG_LINE_PREFIX] [-d DATABASE] [-U USERNAME][--start_time] [--sql_amount] [--statement] [--json] [--max_reserved_period] [--max_template_num] ``` 其中的输入参数依次为：