diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/ai-capabilities/1-predictor-ai-query-time-forecasting.md b/product/en/docs-mogdb/v2.1/characteristic-description/ai-capabilities/1-predictor-ai-query-time-forecasting.md new file mode 100644 index 0000000000000000000000000000000000000000..967ddb0424770f47df5a3c8d656acf8467f8016a --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/ai-capabilities/1-predictor-ai-query-time-forecasting.md @@ -0,0 +1,48 @@ +--- +title: Predictor AI Query Time Forecasting +summary: Predictor AI Query Time Forecasting +author: Guo Huan +date: 2022-05-09 +--- + +# Predictor: AI Query Time Forecasting + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +Predictor is a query time forecasting tool that leverages machine learning and has online learning capability. Predictor can predict the execution time of a plan in the database kernel by continuously learning the historical execution information collected in the database. + +## Benefits + +This feature predicts the SQL statement execution time based on the AI model before the SQL statement is executed. Based on the estimated SQL time, you can detect possible abnormal SQL statements and optimize or schedule them in advance to prevent system running from being affected. + +## Description + +The prediction of AI query time depends on the collection of local query plans. During query execution, you need to collect the actual query plan (including the plan structure, operator type, related data source, and filter criteria), actual execution time of each operator node, cost estimated by the optimizer, number of rows returned by the optimizer, number of rows estimated by the optimizer, and number of parallel rows. These records are stored in data tables and managed persistently, and expired data will be cleared periodically. + +After the database kernel automatically collects historical data, the administrator encodes the data and sends a request to Python through the CURL API to configure the model, send data, and trigger training. You can call the TensorBoard API to monitor the training process. The model returns the prediction accuracy of each project and saves the final model. The database updates the system tables related to the model information based on the result returned by the AI engine. + +This feature is triggered when **explain (analyze on, predictor ) SELECT… “** is executed and the model is converged. The database determines whether the current model meets the requirements based on the plan encoding. If the model meets the requirements, the model loading request is sent to Python (the model needs to be loaded only once after the model training is complete). After the model is loaded, the plan encoding file is inferred and the prediction result of each node is returned. + +## Enhancements + +None + +## Constraints + +- The database system is normal. The user successfully logs in to the database through identity authentication and accesses the authorized data. +- The SQL syntax is correct and no error is reported. +- SQL statements executed by users do not cause database exceptions. +- In the historical performance data window, the cluster concurrency is stable, the cluster scale, table structure, and table quantity remain unchanged, the data volume does not change abruptly, and the GUC parameters related to query performance remain unchanged. If the preceding conditions are damaged, the model becomes invalid. In this case, you need to invalidate all historical performance data, collect data again, and retrain the model. +- When a model is loaded, the structure, parameters, and training information of the original model are completely saved. If the original model information is lost, the model cannot be loaded and needs to be trained again. +- Historical performance data can be properly collected and encoded, and no error is reported when the encoded data is properly parsed. +- You can install the following software by using the provided installation scripts or by yourself: Python\==3.6.4, configparser\==3.8.1, Flask\==0.12.2, Keras\==2.2.4, numpy\==1.16.4, scikit-learn\==0.19.1, pandas\==0.25.1, tensorboard\==1.14.0, and tensorflow-gpu\==1.14.0 or tensorflow\==1.14.0. +- OpenSSL has been installed in the user environment, and the certificate has been generated using a script or based on the usage description. +- Currently, administrators need to manually synchronize models across database namespaces. Automatic synchronization is not supported. + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/ai-capabilities/2-x-tuner-parameter-optimization-and-diagnosis.md b/product/en/docs-mogdb/v2.1/characteristic-description/ai-capabilities/2-x-tuner-parameter-optimization-and-diagnosis.md new file mode 100644 index 0000000000000000000000000000000000000000..c92118ef62cb394a894803aadca7641fb5e23e0f --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/ai-capabilities/2-x-tuner-parameter-optimization-and-diagnosis.md @@ -0,0 +1,46 @@ +--- +title: X-Tuner Parameter Optimization and Diagnosis +summary: X-Tuner Parameter Optimization and Diagnosis +author: Guo Huan +date: 2022-05-09 +--- + +# X-Tuner: Parameter Optimization and Diagnosis + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +X-Tuner is a parameter tuning tool integrated into databases. It uses AI technologies such as deep reinforcement learning and global search algorithms to obtain the optimal database parameter settings without manual intervention. This function is not forcibly deployed with the database environment. It can be independently deployed and run without the database installation environment. + +## Benefits + +This tool can quickly provide the parameter adjustment configuration of the current load in any scenario, reducing database administrator's manual intervention, improving the O&M effect, and meeting customer expectations. + +## Description + +X-Tuner can run in any of the following modes: + +- **recommend**: Log in to the database using the specified user name, obtain the feature information about the running workload, and generate a parameter recommendation report based on the feature information. Report improper parameter settings and potential risks in the current database. Output the currently running workload behavior and characteristics. Output the recommended parameter settings. In this mode, the database does not need to be restarted. In other modes, the database may need to be restarted repeatedly. +- **train**: Modify parameters and execute the benchmark based on the benchmark information provided by users. The reinforcement learning model is trained through repeated iteration so that you can load the model in **tune** mode for optimization. +- **tune**: Use an optimization algorithm to tune database parameters. Currently, two types of algorithms are supported: deep reinforcement learning and global search algorithm (global optimization algorithm). The deep reinforcement learning mode requires **train** mode to generate the optimized model after training. However, the global search algorithm does not need to be trained in advance and can be directly used for search and optimization. + +## Enhancements + +None + +## Constraints + +- The database is normal, the client can be properly connected, and data can be imported to the database. As a result, the optimization program can perform the benchmark test for optimization effect. +- To use this tool, you need to specify the user who logs in to the database. The user who logs in to the database must have sufficient permissions to obtain sufficient database status information. +- If you log in to the database host as a Linux user, add *$GAUSSHOME***/bin** to the _PATH_environment variable so that you can directly run database O&M tools, such as gsql, gs_guc, and gs_ctl. +- The recommended Python version is Python 3.6 or later. The required dependency has been installed in the operating environment, and the optimization program can be started properly. You can install a Python 3.6+ environment independently without setting it as a global environment variable. You are not advised to install the tool as the root user. If you install the tool as the root user and run the tool as another user, ensure that you have the read permission on the configuration file. +- This tool can run in three modes. In **tune** and **train** modes, you need to configure the benchmark running environment and import data. This tool will iteratively run the benchmark to check whether the performance is improved after the parameters are modified. +- In **recommend** mode, you are advised to run the command when the database is executing the workload to obtain more accurate real-time workload information. +- By default, this tool provides benchmark running script samples of TPC-C, TPC-H, TPC-DS, and sysbench. If you use the benchmarks to perform pressure tests on the database system, you can modify or configure the preceding configuration files. To adapt to your own service scenarios, you need to compile the script file that drives your customized benchmark based on the **template.py** file in the **benchmark** directory. + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/ai-capabilities/3-sqldiag-slow-sql-discovery.md b/product/en/docs-mogdb/v2.1/characteristic-description/ai-capabilities/3-sqldiag-slow-sql-discovery.md new file mode 100644 index 0000000000000000000000000000000000000000..a4c27262ad4191266a3290b8ab78026c15286f6b --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/ai-capabilities/3-sqldiag-slow-sql-discovery.md @@ -0,0 +1,39 @@ +--- +title: SQLdiag Slow SQL Discovery +summary: SQLdiag Slow SQL Discovery +author: Guo Huan +date: 2022-05-09 +--- + +# SQLdiag: Slow SQL Discovery + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +SQLdiag is an SQL statement execution time prediction tool. It predicts the execution time of SQL statements based on the statement logic similarity and historical execution records without obtaining the SQL statement execution plan using a template. + +## Benefits + +- The tool does not require users to provide SQL execution plans. Therefore, the database performance is not affected. +- Different from other algorithms in the industry that are limited to OLAP or OLTP, this tool is more widely used. + +## Description + +The SQLdiag focuses on the historical SQL statements of the database, summarizes the execution performance of the historical SQL statements, and then uses the historical SQL statements to infer unknown services. The execution duration of SQL statements in the database does not differ greatly in a short period of time. SQLdiag can detect the statement result set similar to the executed SQL statements from historical data and predict the execution duration of SQL statements based on the SQL vectorization technology and template-based method. + +## Enhancements + +None + +## Constraints + +- The historical logs and the format of the workload to be predicted meet the requirements. You can use the GUC parameter of the database to enable the collection or use the monitoring tool to collect logs. +- To ensure the prediction accuracy, the historical statement logs provided by users should be as comprehensive and representative as possible. +- The Python environment has been configured as required. + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/ai-capabilities/4-anomaly-detection-database-indicator-collection-forecasting-and-exception-monitoring.md b/product/en/docs-mogdb/v2.1/characteristic-description/ai-capabilities/4-anomaly-detection-database-indicator-collection-forecasting-and-exception-monitoring.md new file mode 100644 index 0000000000000000000000000000000000000000..12abf74e1a22e66918dd6a77933218fd8feaf6f1 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/ai-capabilities/4-anomaly-detection-database-indicator-collection-forecasting-and-exception-monitoring.md @@ -0,0 +1,44 @@ +--- +title: Anomaly-detection Database Indicator Collection, Forecasting, and Exception Monitoring +summary: Anomaly-detection Database Indicator Collection, Forecasting, and Exception Monitoring +author: Guo Huan +date: 2022-05-09 +--- + +# Anomaly-detection: Database Indicator Collection, Forecasting, and Exception Monitoring + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +Anomaly_detection is an AI tool integrated into MogDB and can be used to collect and predict database indicators, as well as monitor and diagnose exceptions. It is a component in the dbmind suite. The following information can be collected: IO_Read, IO_Write, CPU_Usage, Memory_Usage, and disk space occupied by the database. Anomaly_detection can monitor multiple indicators at the same time and predict the change trend of each indicator. When detecting that an indicator exceeds the manually set threshold in a certain period or at a certain moment in the future, the tool generates an alarm through logs. + +## Benefits + +- This greatly simplifies the work of O&M personnel, releases a large number of labor resources, and reduces costs for the company. +- This feature helps users detect exceptions in advance and prevent database exceptions from causing greater loss. + +## Description + +Anomaly_detection consists of agent and detector. The agent and MogDB database are deployed on the same server. The agent module provides the following functions: Periodically collect database indicator data and store the collected data in the buffer queue. Periodically send the data in the buffer queue to the detector. + +The detector module communicates with the agent module based on HTTP or HTTPS. Therefore, the detector module can be deployed on any server that can communicate with the agent module. The detector module has the following functions: Receive the data sent by the agent and cache the collected data locally. Predict the future change trend of the indicator and report alarms based on the collected database indicator data. + +## Enhancements + +None + +## Constraints + +- The database is normal, and the data directory has been written into environment variables and named **PGDATA**. +- If you log in to the database host as a Linux user, add \$GAUSSHOME/bin to the _PATH_environment variable so that you can directly run database O&M tools, such as gsql, gs_guc, and gs_ctl. +- The recommended Python version is Python 3.6 or later. The required dependency has been installed in the operating environment, and the optimization program can be started properly. +- This tool consists of the agent and detector. Data is transmitted between the agent and detector in HTTP or HTTPS mode. Therefore, ensure that the agent server can communicate with the detector server properly. +- Detector module runs the server and monitor services, which need to be started separately. +- If HTTPS is used for communication, you need to prepare the CA certificate, and certificates and keys of the agent and detector, and save them to **ca**, **agent**, and **server** in the **root** directory of the project, respectively. In addition, you need to save the key encryption password to **pwf** of the certificate, and set the permission to **600** to prevent other users from performing read and write operations. You can also use the script in the **share** directory to generate certificates and keys. + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/ai-capabilities/5-index-advisor-index-recommendation.md b/product/en/docs-mogdb/v2.1/characteristic-description/ai-capabilities/5-index-advisor-index-recommendation.md new file mode 100644 index 0000000000000000000000000000000000000000..d64d3e2473c18e71b927b9aeecd921f386bafb12 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/ai-capabilities/5-index-advisor-index-recommendation.md @@ -0,0 +1,40 @@ +--- +title: Index-advisor Index Recommendation +summary: Index-advisor Index Recommendation +author: Guo Huan +date: 2022-05-09 +--- + +# Index-advisor: Index Recommendation + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +Index-advisor is an intelligent database index recommendation tool that covers multiple task levels and application scenarios. It provides the single-query index recommendation function, virtual index function, and workload-level index recommendation function to provide reliable index suggestions for users. + +## Benefits + +This feature provides the quick and reliable index recommendation function, greatly simplifying the work of O&M personnel. + +## Description + +The single-query index recommendation function allows users to directly perform operations in the database. This function generates recommended indexes for a single query statement entered by users based on the semantic information of the query statement and the statistics of the database. The virtual index function allows users to directly perform operations in the database. This function simulates the creation of a real index to avoid the time and space overhead required for creating a real index. Based on the virtual index, users can evaluate the impact of the index on the specified query statement by using the optimizer. The workload-level index recommendation can be used by running scripts outside the database. This function uses the workload of multiple DML statements as the input to generate a batch of indexes that can optimize the overall workload execution performance. + +## Enhancements + +None + +## Constraints + +The database is normal, and the client can be connected properly. + +The gsql tool has been installed by the current user, and the tool path has been added to the _PATH_environment variable. + +The Python 3.6+ environment is available. + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/ai-capabilities/6-deepsql-ai-algorithm-in-the-database.md b/product/en/docs-mogdb/v2.1/characteristic-description/ai-capabilities/6-deepsql-ai-algorithm-in-the-database.md new file mode 100644 index 0000000000000000000000000000000000000000..d5284c0b51566d5011e027548effde8290270942 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/ai-capabilities/6-deepsql-ai-algorithm-in-the-database.md @@ -0,0 +1,38 @@ +--- +title: DeepSQL AI Algorithm in the Database +summary: DeepSQL AI Algorithm in the Database +author: Guo Huan +date: 2022-05-09 +--- + +# DeepSQL: AI Algorithm in the Database + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +The database DeepSQL feature implements the DB4AI function, that is, the AI algorithm is implemented in the database to better support quick analysis and computing of big data. A complete set of SQL-based machine learning, data mining, and statistics algorithms is provided. Users can directly use SQL statements to perform machine learning. DeepSQL can abstract the end-to-end R&D process from data to models. With the bottom-layer engine and automatic optimization, technical personnel with basic SQL knowledge can complete most machine learning model training and prediction tasks. The entire analysis and processing are running in the database engine. Users can directly analyze and process data in the database without transferring data between the database and other platforms. This avoids unnecessary data movement between multiple environments. + +## Benefits + +Users can directly use AI algorithms in the database to avoid extra costs caused by migration of a large amount of data. In addition, models can be centrally managed by the database, which is easy to use. + +## Description + +DeepSQL is an enhancement to MogDB DB4AI. DeepSQL encapsulates common machine learning algorithms into UDF and supports more than 60 general algorithms, including regression algorithms (such as linear regression, logistic regression, and random forest), classification algorithms (such as KNN), and clustering algorithms (such as K-means). In addition to basic machine learning algorithms, graph-related algorithms are also included, such as algorithms about the shortest path and graph diameter. Also, it supports data processing (such as PCA), sparse vectors, common statistical algorithms (such as covariance and Pearson coefficient calculation), training set and test set segmentation, and cross validation. + +## Enhancements + +None + +## Constraints + +- Python 2.7.12 or later has been installed. +- The database supports the PL/Python stored procedure. +- You have the administrator permission to install the algorithm database. + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/application-development-interfaces/1-standard-sql.md b/product/en/docs-mogdb/v2.1/characteristic-description/application-development-interfaces/1-standard-sql.md new file mode 100644 index 0000000000000000000000000000000000000000..c5cb1e72f192ac81578004234e8a72a83d8f4e11 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/application-development-interfaces/1-standard-sql.md @@ -0,0 +1,38 @@ +--- +title: Standard SQL +summary: Standard SQL +author: Guo Huan +date: 2022-05-09 +--- + +# Standard SQL + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +SQL is a standard computer language used to control the access to databases and manage data in databases. SQL standards are classified into core features and optional features. Most databases do not fully support SQL standards. + +MogDB supports most of the core features of SQL:2011 and some optional features, providing a unified SQL interface for users. + +## Benefits + +All database vendors can use a unified SQL interface, reducing the costs of learning languages and migrating applications. + +## Description + +For details, see “SQL Syntax” in the *Reference Guide*. + +## Enhancements + +None + +## Constraints + +None + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/application-development-interfaces/2-standard-development-interfaces.md b/product/en/docs-mogdb/v2.1/characteristic-description/application-development-interfaces/2-standard-development-interfaces.md new file mode 100644 index 0000000000000000000000000000000000000000..6ab0f9bf209e18426bb816d5ba3b223b1c6595bb --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/application-development-interfaces/2-standard-development-interfaces.md @@ -0,0 +1,36 @@ +--- +title: Standard Development Interfaces +summary: Standard Development Interfaces +author: Guo Huan +date: 2022-05-09 +--- + +# Standard Development Interfaces + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +Standard ODBC 3.5 and JDBC 4.0 APIs are supported. + +## Benefits + +Standard ODBC and JDBC interfaces are provided to ensure quick migration of user services to MogDB. + +## Description + +Currently, the standard ODBC 3.5 and JDBC 4.0 APIs are supported. The ODBC interface supports SUSE Linux, Windows 32-bit, and Windows 64-bit platforms. The JDBC API supports all platforms. + +## Enhancements + +The function of connecting JDBC to a third-party log framework is added. JDBC can interconnect with a third-party log framework to meet users' log management and control requirements. + +## Constraints + +None + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/application-development-interfaces/3-postgresql-api-compatibility.md b/product/en/docs-mogdb/v2.1/characteristic-description/application-development-interfaces/3-postgresql-api-compatibility.md new file mode 100644 index 0000000000000000000000000000000000000000..17862197f566fe2a39107abf5fce7eaf05b50fad --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/application-development-interfaces/3-postgresql-api-compatibility.md @@ -0,0 +1,36 @@ +--- +title: PostgreSQL API Compatibility +summary: PostgreSQL API Compatibility +author: Guo Huan +date: 2022-05-09 +--- + +# PostgreSQL API Compatibility + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +Compatible with PostgreSQL clients and standard APIs. + +## Benefits + +Compatible with the PostgreSQL clients and standard APIs, and can be seamlessly interconnected with PostgreSQL ecosystem tools. + +## Description + +Compatible with PostgreSQL clients and standard APIs. + +## Enhancements + +None + +## Constraints + +None + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/application-development-interfaces/4-pl-java.md b/product/en/docs-mogdb/v2.1/characteristic-description/application-development-interfaces/4-pl-java.md new file mode 100644 index 0000000000000000000000000000000000000000..50b2866445a1821ac45c0e3aa260c8e6b94d096f --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/application-development-interfaces/4-pl-java.md @@ -0,0 +1,66 @@ +--- +title: PL/Java +summary: PL/Java +author: Guo Huan +date: 2022-05-09 +--- + +# PL/Java + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +The Java user-defined field (UDF) is supported. + +## Benefits + +A development environment is provided for multiple functions. + +## Description + +With the MogDB PL/Java function, you can choose your favorite Java IDE to write Java methods and install the JAR files containing these methods into MogDB before invoking them. MogDB PL/Java is developed based on open-source tada PL/Java 1.5.2. PL/Java uses Huawei JDK V100R001C00SPC190B003-b09. + +## Enhancements + +None + +## Constraints + +- Java UDFs can implement simple Java computing. However, do not encapsulate services in Java UDFs. + +- Do not connect to a database in any way (for example, JDBC) in Java functions. + +- Use Huawei JDK V100R001C00SPC190B003-b09 to compile Java methods and JAR files. + +- Currently, only data types listed in Table 1 are supported. Other data types, such as user-defined data types and complex data types (for example, Java array and its derived types) are not supported. + +- Currently, UDAF and UDTF are not supported. + + **Table 1** PL/Java mapping for default data types + + | MogDB | **Java** | + | :---------- | :------------------------------------------------- | + | BOOLEAN | boolean | + | “char” | byte | + | bytea | byte[] | + | SMALLINT | short | + | INTEGER | int | + | BIGINT | long | + | FLOAT4 | float | + | FLOAT8 | double | + | CHAR | java.lang.String | + | VARCHAR | java.lang.String | + | TEXT | java.lang.String | + | name | java.lang.String | + | DATE | java.sql.Timestamp | + | TIME | java.sql.Time (stored value treated as local time) | + | TIMETZ | java.sql.Time | + | TIMESTAMP | java.sql.Timestamp | + | TIMESTAMPTZ | java.sql.Timestamp | + +## Dependencies + +PL/Java depends on the Java Development Kit (JDK) environment. Currently, JDK is included in MogDB and installation is not required. If you have installed the same or different versions of JDK, no conflict will occur. MogDB uses Huawei JDK V100R001C00SPC190B003-b09 to run PL/Java. \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/database-security/1-access-control-model.md b/product/en/docs-mogdb/v2.1/characteristic-description/database-security/1-access-control-model.md new file mode 100644 index 0000000000000000000000000000000000000000..e34d585c8363e31c636ead161ce63b9c9f738725 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/database-security/1-access-control-model.md @@ -0,0 +1,42 @@ +--- +title: Access Control Model +summary: Access Control Model +author: Guo Huan +date: 2022-05-09 +--- + +# Access Control Model + +## Availability + +This feature is available as of MogDB 1.1.0. + +## Introduction + +The access control model can be used to manage users' access permissions and grant them the minimum permissions required for completing a task. + +## Benefits + +You can create users and grant permissions to them as needed to minimize risks. + +## Description + +The database provides a role-based access control model and an access control model based on the separation of duties. In the role-based access control model, database roles are classified into system administrator, monitoring administrator, O&M administrator, security policy administrator, and common user. The security administrator creates roles or user groups and grant permissions to roles. The monitoring administrator views the monitoring views or functions in **dbe_perf** mode. The O&M administrator uses the Roach tool to back up and restore the database. The security policy administrator creates resource labels, anonymization policies, and unified audit policies. A user who is assigned a role has the role's permissions. + +In the access control model based on the separation of duties, database roles are classified into system administrator, security administrator, audit administrator, monitoring administrator, O&M administrator, security policy administrator, and common user. The security administrator creates users, the system administrator grants permissions to users, and the audit administrator audits all user behavior. + +By default, the role-based access control model is used. To switch to another mode, set the GUC parameter **enableSeparationOfDuty** to **on**. + +## Enhancements + +None + +## Constraints + +The permissions of the system administrator are controlled by the GUC parameter **enableSeparationOfDuty**. + +The database needs to be restarted when the separation of duties is enabled, disabled or switched. In addition, improper user permissions in the new model cannot be automatically identified. The database administrator needs to manually identify and rectify the fault. + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/database-security/10-row-level-access-control.md b/product/en/docs-mogdb/v2.1/characteristic-description/database-security/10-row-level-access-control.md new file mode 100644 index 0000000000000000000000000000000000000000..88b39f5a16c1bc0d835e9562d8e69e8040ccdf73 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/database-security/10-row-level-access-control.md @@ -0,0 +1,43 @@ +--- +title: Row-Level Access Control +summary: Row-Level Access Control +author: Guo Huan +date: 2022-05-09 +--- + +# Row-Level Access Control + +## Availability + +This feature is available as of MogDB 1.1.0. + +## Introduction + +The row-level access control feature enables database access control to be accurate to each row of data tables. When different users perform the same SQL query operation, the read results may be different. + +## Benefits + +When different users perform the same SQL query operation, the read results may be different. + +## Description + +You can create a row-level access control policy for a data table. The policy defines an expression that takes effect only for specific database users and SQL operations. When a database user accesses the data table, if a SQL statement meets the specified row-level access control policy of the data table, the expressions that meet the specified condition will be combined by using **AND** or **OR** based on the attribute type (**PERMISSIVE** or **RESTRICTIVE**) and applied to the execution plan in the query optimization phase. + +Row-level access control is used to control the visibility of row-level data in tables. By predefining filters for data tables, the expressions that meet the specified condition can be applied to execution plans in the query optimization phase, which will affect the final execution result. Currently, the SQL statements that can be affected include **SELECT**, **UPDATE**, and **DELETE**. + +## Enhancements + +None + +## Constraints + +- Row-level access control policies can be applied only to **SELECT**, **UPDATE**, and **DELETE** operations and cannot be applied to **INSERT** and **MERGE** operations. +- Row-level access control policies can be defined for row-store tables, row-store partitioned tables, column-store tables, column-store partitioned tables, replication tables, unlogged tables, and hash tables. Row-level access control policies cannot be defined for HDFS tables, foreign tables, and temporary tables. +- Row-level access control policies cannot be defined for views. +- A maximum of 100 row-level access control policies can be defined for a table. +- Initial users and system administrators are not affected by row-level access control policies. +- If a dynamic data anonymization policy is configured for a table that has the row-level access control policies defined, grant the trigger permission of the table to other users with caution to prevent other users from using the trigger to bypass the anonymization policy. + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/database-security/11-password-strength-verification.md b/product/en/docs-mogdb/v2.1/characteristic-description/database-security/11-password-strength-verification.md new file mode 100644 index 0000000000000000000000000000000000000000..de840e345f000f3ac77d4c3120546c4d3dc9ac9e --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/database-security/11-password-strength-verification.md @@ -0,0 +1,70 @@ +--- +title: Password Strength Verification +summary: Password Strength Verification +author: Guo Huan +date: 2022-05-09 +--- + +# Password Strength Verification + +## Availability + +This feature is available as of MogDB 1.1.0. + +## Introduction + +Verifies the password strength when users access the database. + +## Benefits + +Users cannot set passwords with low strength to enhance customer data security. + +## Description + +You need to specify a password when initializing a database, creating a user, or modifying a user. The password must meet the strength requirements. Otherwise, the system prompts you to enter the password again. Password complexity requirements: + +- Minimum number of uppercase letters (A-Z) (**password_min_uppercase**) +- Minimum number of lowercase letters (a-z) (**password_min_lowercase**) +- Minimum number of digits (0-9) (**password_min_digital**) +- Minimum number of special characters (**password_min_special**) +- Minimum password length (**password_min_length**) +- Maximum password length (**password_max_length**) +- A password must contain at least three types of the characters (uppercase letters, lowercase letters, digits, and special characters). +- A password is case insensitive and cannot be the username or the username spelled backwards. +- A new password cannot be the current password and the current password spelled backwards. +- It must be a strong password. + - Weak passwords are weak passwords that are easy to crack. The definition of weak passwords may vary with users or user groups. Users can define their own weak passwords. + +If parameter **password_policy** is set to **1**, the default password complexity rule is used to check passwords. + +Passwords in the weak password dictionary are stored in the **gs_global_config** system catalog (the record whose name field is **weak_password** is the stored weak password). When a user is created or modified, the password set by the user is compared with the password stored in the weak password dictionary. If the password is matched, a message is displayed, indicating that the password is weak and the password fails to be set. + +The weak password dictionary is empty by default. You can add or delete weak passwords using the following syntax: + +``` +CREATE WEAK PASSWORD DICTIONARY WITH VALUES ('password1'), ('password2'); +DROP WEAK PASSWORD DICTIONARY; +``` + +In the preceding statement, **password1** and **password2** are weak passwords prepared by users. After the statement is executed successfully, the passwords are saved to the weak password system catalog. + +When a user attempts to run the CREATE WEAK PASSWORD DICTIONARY statement to insert a weak password that already exists in the table, only one weak password is retained in the table. + +The DROP WEAK PASSWORD DICTIONARY statement clears weak passwords in the entire system catalog. + +The gs_global_config system catalog does not have a unique index. You are not advised to use the COPY FROM statement to copy the same data to the gs_global_config system catalog. + +To audit weak password operations, set the third bit of the value of the **audit_system_object** parameter to **1**. + +## Enhancements + +In V500R001C10, the weak password dictionary function is implemented. + +## Constraints + +- Initial users, system administrators, and security administrators can view, add, and delete weak password dictionaries. +- Common users can view but cannot add or delete weak password dictionaries. + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/database-security/12-equality-query-in-a-fully-encrypted-database.md b/product/en/docs-mogdb/v2.1/characteristic-description/database-security/12-equality-query-in-a-fully-encrypted-database.md new file mode 100644 index 0000000000000000000000000000000000000000..18df859290594e69a52d2dc1421b443cccf349e1 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/database-security/12-equality-query-in-a-fully-encrypted-database.md @@ -0,0 +1,108 @@ +--- +title: Equality Query in a Fully-encrypted Database +summary: Equality Query in a Fully-encrypted Database +author: Guo Huan +date: 2022-05-09 +--- + +# Equality Query in a Fully-encrypted Database + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +The encrypted database aims to protect privacy throughout the data lifecycle. In this way, data is always in ciphertext during transmission, computing, and storage regardless of the service scenario and environment. After the data owner encrypts data on the client and sends the encrypted data to the server, no attacker can obtain valuable information even if the attacker steals user data by exploiting system vulnerabilities. In this way, data privacy is protected. + +## Benefits + +The entire service data flow is in ciphertext during data processing, so the following can be implemented by using a fully-encrypted database: + +1. Protect data privacy and security throughout the lifecycle on the cloud. Attackers cannot obtain valid information from the database server regardless of the data status. +2. Help cloud service providers obtain third-party trust. Users, including service administrators and O&M administrators in enterprise service scenarios and application developers in consumer cloud services, can keep keys by themselves so that users with high permissions cannot obtain valid data. +3. Enable cloud databases to better comply with personal privacy protection laws and regulations with the help of the fully-encrypted database. + +## Description + +From the perspective of users, the encrypted equality query functions are divided into three parts, which are implemented by the newly added KeyTool and the enhanced MogDB gsql client. + +First, this feature provides the client key management function. Users can use KeyTool to generate, destroy, and update CMKs, and import and export keys. With the import and export functions of KeyTool, CMKs can be transmitted between different clients. In addition, the KeyTool implements key management on a single client. By configuring management files, you can store and update keys. + +In addition, this feature provides the key creation and encrypted table creation functions. The SQL syntax CREATE CLINET MASTER KEY and CREATE COLUMN ENCRYPTION KEY are added to record and manage CMK and CEK metadata in the database. The CMK and CEK information is recorded in the new system catalog. The CREATE TABLE syntax is extended to specify a column encryption key and encryption algorithm for each sensitive information column in a table, facilitating subsequent ciphertext data storage. + +This feature supports the encrypted equality query function, which is the core of the entire feature. Although users are unaware of the ciphertext query, the query of sensitive data is restricted by the specifications of the current encrypted equality query. + +From the overall perspective, this feature is used to store and manage data based on sensitive data protection requirements and implement query tasks based on ciphertext data. + +## Enhancements + +None. + +## Constraints + +- Data is encrypted at the column level, and encryption policies cannot be differentiated by row level. + +- Except the RENAME operation, the ALTER TABLE syntax cannot be used to change columns in an encrypted table (including the conversion between encrypted and unencrypted columns). The ADD and DROP operations can be used to add and delete encrypted columns, respectively. + +- The CHECK(COLUMN IS NOT NULL) syntax can be used, but most check constraint syntax cannot be set for encrypted columns. + +- When **support_extended_features** is set to **off**, primary key and unique cannot be used for encrypted columns. When **support_extended_features** is set to **on**, only primary key and unique can be used for encrypted columns. + +- Different data types cannot be implicitly converted. + +- The set operation cannot be performed between ciphertexts of different data types. + +- Range partitioning cannot be created for encrypted columns. + +- Only the repeat and empty_blob() functions can be used to encrypt columns. + +- The current version supports only gsql and JDBC (deployed on a Linux OS) clients. Other clients such as ODBC do not support encrypted equality query. + +- Data can only be imported to the encrypted table by running **copy from stdin**, **\copy**, or **insert into values (…)** on the client. + +- Copying an encrypted table to a file is not supported. + +- The system does not support encrypted queries, such as sorting, range query, and fuzzy query, except equality query. + +- The encrypted syntax of stored procedures for some functions is supported. For details about the constraints, see “Encrypted Functions and Stored Procedures” in the *Developer Guide*. + +- Non-encrypted table data cannot be inserted into encrypted table data using the **INSERT INTO… SELECT…** or **MERGE INTO** syntax. + +- For a request in connection state, the CEK information change on the server can be detected only after the cache update operation is triggered (for example, the user is changed or the encrypted column fails to be decrypted) and the connection is re-established. + +- Encrypted equality query is not supported on columns encrypted using the random encryption algorithm. + +- An error is reported if the two attribute conditions used for comparison in the encrypted equality query use different data encryption keys. + +- Encrypted equality query is not supported in time series tables and foreign tables. The ustore storage engine is not supported. + +- If the database service configuration (such as the pg_settings system catalog, permission, key, and encrypted column) is changed, you need to re-establish a JDBC connection to make the configuration take effect. + +- Multiple SQL statements cannot be executed at the same time. This constraint does not apply to the scenario where the INSERT INTO statement is executed in multiple batches. + +- Encrypted equality query supports the following data types: + + | Category | Type | Description | + | -------------------- | ------------------ | ------------------------------------------------------------ | + | Integer types | tinyint/tinyint(n) | Tiny integer, which is the same as int1. | + | | smallint | Small integer, which is the same as int2. | + | | int4 | Common integer. | + | | binary_integer | Oracle compatibility type. Generally, the value is an integer. | + | | bigint/bigint(n) | Big integer, which is the same as int8. | + | Numeric data types | numeric(p,s) | A number with the precision **p**. | + | | number | Oracle compatibility type, which is the same as numeric(p,s). | + | Floating point types | float4 | Single-precision floating point. | + | | float8 | Double-precision floating point. | + | | double precision | Double-precision floating point. | + | Character data types | char/char(n) | Fixed-length character string. If the length is insufficient, add spaces. The default precision is **1**. | + | | varchar(n) | Variable-length character string, where **n** indicates the maximum number of bytes. | + | | text | Text type. | + | | varchar2(n) | Oracle compatibility type, which is the same as varchar(n). | + | | clob | Character large object. | + | Binary data types | bytea | Variable-length binary string. | + | | blob | Binary large object. | + +## Dependencies + +None. \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/database-security/13-ledger-database-mechanism.md b/product/en/docs-mogdb/v2.1/characteristic-description/database-security/13-ledger-database-mechanism.md new file mode 100644 index 0000000000000000000000000000000000000000..c7d63121b7357258cca8b6129b3cc2191af52d69 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/database-security/13-ledger-database-mechanism.md @@ -0,0 +1,43 @@ +--- +title: Ledger Database Mechanism +summary: Ledger Database Mechanism +author: Guo Huan +date: 2022-05-09 +--- + +# Ledger Database Mechanism + +## Availability + +This feature is available since MogDB 2.1.0. + +## Introduction + +The ledger database feature includes adding the verification information to a tamper-proof table specified by a user and recording the user's data operation history. The consistency between the data and operation history is checked to ensure that the user data cannot be maliciously tampered with. When a user performs DML operations on a tamper-proof table, the system adds a small amount of additional row-level verification information to the table and records the SQL statements and data change history. The feature provides a verification API for users to check whether the data in the tamper-proof table is consistent with the operation information recorded by the system. + +## Benefits + +The ledger database provides user data operation records, historical data change records, and easy-to-use consistency verification API to help users check whether sensitive information in the database is maliciously tampered with at any time, effectively improving the tamper-proof capability of the database. + +## Description + +The ledger database uses the ledger schema to isolate common tables from tamper-proof user tables. If a row-store table created in the ledger schema has the tamper-proof attribute, it is a tamper-proof user table. When data is inserted into a tamper-proof user table, the system automatically generates a small amount of row-level verification information. When a user executes DML, the system records user operations in the global blockchain table (GS_GLOBAL_CHAIN) and records data changes in the historical table corresponding to the user table. The data in operation records, data change records, and the user table must be the same. The ledger database provides a high-performance verification API for users to verify data consistency. If the consistency verification fails, the data may be tampered with. In this case, contact the audit administrator to trace the operation history. + +## Enhancements + +None. + +## Constraints + +- In tamper-proof schema, row-store tables are tamper-proofing, whereas temporary tables, unlogged tables, column-store tables, and time series tables are not. +- The structure of the tamper-proof user table cannot be modified. The tamper-proof tables cannot be truncated. The tamper-proof user table cannot be switched to a common schema. The non-tamper-proof table cannot be switched to the tamper-proof schema. +- If the tamper-proof table is a partitioned table, operations such as exchange partition, drop partition and truncate partition are not supported. +- Functions and triggers cannot be used to modify data in a tamper-proof user table. +- When a tamper-proof user table is created, the column named **hash** cannot exist. +- Common users can call the tampering verification API to verify only tables that they have the permission to query. +- Only the audit administrator and initial user can query the global blockchain table and tables in BLOCKCHAIN schema. Common users do not have the permission to access and all users do not have the permission to modify the tables. +- According to the naming rules of historical tables, if the name of the schema or table to be created ends or starts with an underscore (_), the name of the corresponding historical table may conflict with that of an existing table. In this case, you need to rename the table. + +## Dependencies + +None. \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/database-security/2-separation-of-control-and-access-permissions.md b/product/en/docs-mogdb/v2.1/characteristic-description/database-security/2-separation-of-control-and-access-permissions.md new file mode 100644 index 0000000000000000000000000000000000000000..4f09ac72869d04afa9e4f443c6c008a9fb3e483a --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/database-security/2-separation-of-control-and-access-permissions.md @@ -0,0 +1,46 @@ +--- +title: Separation of Control and Access Permissions +summary: Separation of Control and Access Permissions +author: Guo Huan +date: 2022-05-09 +--- + +# Separation of Control and Access Permissions + +## Availability + +This feature is available as of MogDB 1.1.0. + +## Introduction + +The control permissions and the access permissions can be separated. + +## Benefits + +The control permissions of database administrators for tables need to be isolated from their access permissions to improve the data security of common users. + +## Description + +If multiple business departments use different database users to perform service operations and a database maintenance department at the same level uses database administrators to perform O&M operations, the business departments may require that database administrators can only perform control operations (**DROP**, **ALTER**, and **TRUNCATE**) and cannot perform access operations (**INSERT**, **DELETE**, **UPDATE**, **SELECT**, and **COPY**) without authorization. That is, the control permissions of database administrators for tables need to be isolated from their access permissions to improve the data security of common users. + +In separation-of-duties mode, a database administrator does not have permissions for the tables in schemas of other users. In this case, database administrators have neither control permissions nor access permissions. This does not meet the requirements of the business departments mentioned above. Therefore, MogDB provides private users to solve the problem. That is, create private users with the **INDEPENDENT** attribute in non-separation-of-duties mode. Users with the CREATEROLE permission or the system administrator permission can create private users or change the attributes of common users to private users. Common users can also change their own attributes to private users. + +``` +MogDB=# CREATE USER user_independent WITH INDEPENDENT IDENTIFIED BY "1234@abc"; +``` + +System administrators can manage (**DROP**, **ALTER**, and **TRUNCATE**) table objects of private users but cannot access (**INSERT**, **DELETE**, **SELECT**, **UPDATE**, **COPY**, **GRANT**, **REVOKE**, and **ALTER OWNER**) the objects before being authorized. + +## Enhancements + +None + +## Constraints + +For a table owned by a private user, grant the trigger permission of the table to other users with caution to prevent other users from using the trigger to view the data of the private user. + +If permissions related to private user tables are granted to non-private users, the system administrator will obtain the same permissions. + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/database-security/3-database-encryption-authentication.md b/product/en/docs-mogdb/v2.1/characteristic-description/database-security/3-database-encryption-authentication.md new file mode 100644 index 0000000000000000000000000000000000000000..09fe970553c6fc8570d3bf5ed02a25b716e84379 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/database-security/3-database-encryption-authentication.md @@ -0,0 +1,36 @@ +--- +title: Database Encryption Authentication +summary: Database Encryption Authentication +author: Guo Huan +date: 2022-05-09 +--- + +# Database Encryption Authentication + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +The client/server (C/S) mode-based client connection authentication mechanism is provided. + +## Benefits + +The unidirectional, irreversible hash encryption algorithm PBKDF2 is used for encryption and authentication, effectively defending against rainbow attacks. + +## Description + +MogDB uses a basic client connection authentication mechanism. After a client initiates a connection request, the server verifies the information and sends the information required for authentication to the client based on the verification result. The authentication information includes the salt, token, and server signature. The client responds to the request and sends the authentication information to the server. The server calls the authentication module to authenticate the client authentication information. The user password is encrypted and stored in the memory. During the entire authentication process, passwords are encrypted for storage and transmission. When the user logs in to the system next time, the hash value is calculated and compared with the key value stored on the server to verify the correctness. + +## Enhancements + +The message processing flow in the unified encryption and authentication process effectively prevents attackers from cracking the username or password by capturing packets. + +## Constraints + +None + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/database-security/4-data-encryption-and-storage.md b/product/en/docs-mogdb/v2.1/characteristic-description/database-security/4-data-encryption-and-storage.md new file mode 100644 index 0000000000000000000000000000000000000000..a21143e1dc022bd7d14a759ce09e5ea4810156b5 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/database-security/4-data-encryption-and-storage.md @@ -0,0 +1,48 @@ +--- +title: Data Encryption and Storage +summary: Data Encryption and Storage +author: Guo Huan +date: 2022-05-09 +--- + +# Data Encryption and Storage + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +Imported data is encrypted before stored. + +## Benefits + +You can use encrypted import interfaces to encrypt sensitive information and store it in a table. + +## Description + +MogDB provides the encryption functions **gs_encrypt_aes128()** and **gs_encrypt()**, and decryption functions **gs_decrypt_aes128()** and **gs_decrypt()**. Before you import data to a certain column in a table, you can use this function to encrypt the data. The function can be called using a statement in the following format: + +``` +gs_encrypt_aes128(column, key), gs_encrypt (decryptstr, keystr, decrypttype) +``` + +In the preceding command, **key** indicates the initial password specified by the user, which is used to derive the encryption key. To encrypt an entire table, you need to write an encryption function for each column. + +If a user with the required permission wants to view specific data, the user can decrypt required columns using the decryption function interface `gs_decrypt_aes128(column, key)`. To invoke the interface, run the following command: + +``` +gs_decrypt_aes128(column, key), gs_decrypt(decryptstr, keystr, decrypttype) +``` + +## Enhancements + +None. + +## Constraints + +None. + +## Dependencies + +None. \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/database-security/5-database-audit.md b/product/en/docs-mogdb/v2.1/characteristic-description/database-security/5-database-audit.md new file mode 100644 index 0000000000000000000000000000000000000000..84e1d4e60e2b984ca3437dc56c57daa1e4b796be --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/database-security/5-database-audit.md @@ -0,0 +1,36 @@ +--- +title: Database Audit +summary: Database Audit +author: Guo Huan +date: 2022-05-09 +--- + +# Database Audit + +## Availability + +This feature is available as of MogDB 1.1.0. + +## Introduction + +Audit logs record user operations performed on database startup and stopping, as well as connection, DDL, DML, and DCL operations. + +## Benefits + +The audit log mechanism enhances the database capability of tracing unauthorized operations and collecting evidence. + +## Description + +Database security is essential for a database system. MogDB writes all user operations in the database into audit logs. Database security administrators can use the audit logs to reproduce a series of events that cause faults in the database and identify unauthorized users, unauthorized operations, and the time when these operations are performed. + +## Enhancements + +None + +## Constraints + +None + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/database-security/6-network-communication-security.md b/product/en/docs-mogdb/v2.1/characteristic-description/database-security/6-network-communication-security.md new file mode 100644 index 0000000000000000000000000000000000000000..44ab498b3f0bb93cd1eba80a8b753961666f79c9 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/database-security/6-network-communication-security.md @@ -0,0 +1,44 @@ +--- +title: Network Communication Security +summary: Network Communication Security +author: Guo Huan +date: 2022-05-09 +--- + +# Network Communication Security + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +To secure the transmission of sensitive data on the Internet, MogDB encrypts communication between the server and the client using the Secure Socket Layer (SSL) protocol. + +## Benefits + +The communication between your client and the server can be secured. + +## Description + +MogDB supports the SSL protocol. The SSL protocol is an application-layer communication protocol with high security, which is mainly used for secure web transmission. SSL contains a record layer and a transport layer. The record-layer protocol determines the encapsulation format of the transport-layer data. The transport-layer security protocol uses X.509 for authentication. The SSL protocol uses asymmetric encryption algorithms to authenticate the identities of communicating parties, and then the two parties exchange symmetric keys as communication keys. The SSL protocol effectively ensures the confidentiality and reliability of the communication between two applications and prevents the communication between a client and a server from being eavesdropped by attackers. + +MogDB also supports the TLS 1.2 protocol. TLS 1.2 is a transport-layer communication protocol with high security. It consists of the TLS Record and TLS Handshake protocols. Each protocol suit has information in multiple formats. The TLS protocol is independent of application-layer protocols. Upper-layer protocols can be transparently distributed on the TLS protocol. The TLS protocol ensures the data confidentiality and integrity for both communication parties. + +## Enhancements + +Checking the strength of certificate signature algorithms: For low-strength signature algorithms, alarms are reported, reminding you to replace the certificate with another certificate containing a high-strength signature algorithm. + +Checking the certificate validity period: If a certificate is about to expire in less than seven days, an alarm is reported, reminding you to replace the certificate on the client. + +Checking certificate permissions: The certificate permissions are verified at the connection setup stage. + +## Constraints + +The formal certificates and keys for servers and clients shall be obtained from the Certificate Authority (CA). Assume the private key and certificate for a server are **server.key** and **server.crt**, the private key and certificate for the client are **client.key** and **client.crt**, and the CA root certificate is **cacert.pem**. + +You need to enable the SSL protocol and configure the certificate and connection mode. + +## Dependencies + +OpenSSL \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/database-security/7-resource-label.md b/product/en/docs-mogdb/v2.1/characteristic-description/database-security/7-resource-label.md new file mode 100644 index 0000000000000000000000000000000000000000..056fb4342474e8fd809ef0b16e6bf3ab191889c2 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/database-security/7-resource-label.md @@ -0,0 +1,49 @@ +--- +title: Resource Label +summary: Resource Label +author: Guo Huan +date: 2022-05-09 +--- + +# Resource Label + +## Availability + +This feature is available as of V500R001C00. + +## Introduction + +Database resources refer to database objects, such as databases, schemas, tables, columns, views, and triggers. The more the database objects are, the more complex the classification management of database resources is. The resource label mechanism is a technology that classifies and labels database resources with certain features to implement resource classification management. After adding labels to some resources in a database, administrators can perform operations such as data audit or anonymization using the labels to implement security management on labeled database resources. + +## Benefits + +Proper resource labels can be used to effectively classify data objects, improve management efficiency, and simplify security policy configuration. To perform unified audit or data anonymization on a group of database resources, the administrator can allocate a resource label to these resources first. The label indicates that the database resources have a certain feature or require unified configuration of a certain policy. The administrator can directly perform operations on the resource label, which greatly reduces the complexity of policy configuration and information redundancy as well as improves management efficiency. + +## Description + +The resource label mechanism selectively classifies resources in the current database. Administrators can use the following SQL syntax to create a resource label and add the label to a group of database resources: + +``` +CREATE RESOURCE LABEL schm_lb ADD SCHEMA(schema_for_label); +CREATE RESOURCE LABEL tb_lb ADD TABLE(schema_for_label.table_for_label); +CREATE RESOURCE LABEL col_lb ADD COLUMN(schema_for_label.table_for_label.column_for_label); +CREATE RESOURCE LABEL multi_lb ADD SCHEMA(schema_for_label), TABLE(table_for_label); +``` + +**schema_for_label**, **table_for_label**, and **column_for_label** indicate the schema, table, and column to be labeled, respectively. The **schm_lb** label is added to schema **schm_for_label**, **tb_lb** is added to table **table_for_label**, **col_lb** is added to column **column_for_label**, and **multi_lb** is added to schema **schm_for_label** and table **table_for_label**. You can perform unified audit or dynamic data anonymization using the configured resource labels, that is, manage all labeled database resources. + +Currently, resource labels support the following database resource types: schema, table, column, view, and function. + +## Enhancements + +None + +## Constraints + +- Resource labels can be created only by a user with the **POLADMIN** and **SYSADMIN** attributes or an initial user. +- Resource labels cannot be created for temporary tables. +- Columns in the same basic table can belong to only one resource tag. + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/database-security/8-unified-audit.md b/product/en/docs-mogdb/v2.1/characteristic-description/database-security/8-unified-audit.md new file mode 100644 index 0000000000000000000000000000000000000000..c12b99221fc486a6ecaff5c396e12b4f3ffda9d8 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/database-security/8-unified-audit.md @@ -0,0 +1,74 @@ +--- +title: Unified Audit +summary: Unified Audit +author: Guo Huan +date: 2022-05-09 +--- + +# Unified Audit + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +The audit mechanism is a security management solution that can effectively deal with the attackers' repudiation. The larger the audit scope is, the more operations can be monitored and the more audit logs are generated, affecting the actual audit efficiency. The unified audit mechanism is a technology that implements efficient security audit management by customizing audit policies. After the administrator defines the audit object and audit behaviors, if the task executed by a user is associated with an audit policy, the corresponding audit behavior is generated and the audit log is recorded. Customized audit policies can cover common user management activities, as well as DDL and DML operations, meeting routine audit requirements. + +## Benefits + +Audit is indispensable for routine security management. When a traditional audit mechanism is used to audit an operation, such as **SELECT**, a large number of audit logs are generated, increasing the I/O of the entire system and affecting the system performance and audit efficiency of administrators. The unified audit mechanism allows you to customize policies for generating audit logs. For example, only the operation that database account **A** queries table **a** is audited. Customized audit greatly reduces the number of generated audit logs, ensuring audit behaviors and reducing the impact on system performance. In addition, customized audit policies can improve the audit efficiency of administrators. + +## Description + +The unified audit mechanism customizes audit behaviors based on resource labels and classifies the supported audit behaviors into the **ACCESS** and **PRIVILEGES** classes. The SQL syntax for creating a complete audit policy is as follows: + +``` +CREATE RESOURCE LABEL auditlabel add table(table_for_audit1, table_for_audit2); +CREATE AUDIT POLICY audit_select_policy ACCESS SELECT ON LABEL(auditlabel) FILTER ON ROLES(usera); +CREATE AUDIT POLICY audit_admin_policy PRIVILEGES ALTER, DROP ON LABEL(auditlabel) FILTER ON IP(local); +``` + +**auditlabel** indicates the resource label in the current audit, which contains two table objects. **audit_select_policy** defines the audit policy for user **usera** to audit the **SELECT** operation on the objects with the **auditlabel** label, regardless of the access source. **audit_admin_policy** defines a local audit policy for **ALTER** and **DROP** operations on the objects with the **auditlabel** label, regardless of the user. If **ACCESS** and **PRIVILEGES** are not specified, all DDL and DML operations on objects with a resource label are audited. If no audit objects are specified, operations on all objects are audited. The addition, deletion, and modification of unified audit policies are also recorded in unified audit logs. + +Currently, unified audit supports the following audit behaviors: + +| **SQL Type** | Supported operations and object types | +| ------------ | ------------------------------------------------------------ | +| DDL | Operations: ALL, ALTER, ANALYZE, COMMENT, CREATE, DROP, GRANT, and REVOKE
SET SHOW
Objects: DATABASE, SCHEMA, FUNCTION, TRIGGER, TABLE, SEQUENCE, FOREIGN_SERVER, FOREIGN_TABLE, TABLESPACE, ROLE/USER, INDEX, VIEW, and DATA_SOURCE | +| DML | Operations: ALL, COPY, DEALLOCATE, DELETE_P, EXECUTE, REINDEX INSERT, REPARE, SELECT, TRUNCATE, and UPDATE | + +## Enhancements + +None. + +## Constraints + +- The unified audit policy must be created by a user with the **POLADMIN** or **SYSADMIN** attribute, or by the initial user. Common users do not have the permission to access the security policy system catalog and system view. + +- The syntax of a unified audit policy applies to either DDL or DML operations. DDL operations and DML operations are mutually exclusive in an audit policy. A maximum of 98 unified audit policies can be configured. + +- Unified audit monitors the SQL statements executed by users on the clients, but does not record the internal SQL statements of databases. + +- In the same audit policy, the same resource tag can be bound to different audit behaviors, and the same behavior can be bound to different resource tags. The ALL operation type includes all operations supported by DDL or DML. + +- A resource label can be associated with different unified audit policies. Unified audit outputs audit information in sequence based on the policies matched by SQL statements. + +- Audit logs of unified audit policies are recorded separately. Currently, no visualized query interfaces are provided. Audit logs depend on the OS service Rsyslog and are archived through the service configuration. + +- In cloud service scenarios, logs need to be stored in the OBS. In hybrid cloud scenarios, you can deploy Elasticsearch to collect logs and perform visualized processing. + +- It is recommended that **APP** in **FILTER** be set to applications in the same trusted domain. Since a client may be forged, a security mechanism must be formed on the client when **APP** is used to reduce misuse risks. Generally, you are not advised to set **APP**. If it is set, pay attention to the risk of client spoofing. + +- Taking an IPv4 address as an example, the following formats are supported: + + | IP Address Format | Example | + | -------------------- | ------------------------ | + | Single IP address | 127.0.0.1 | + | IP address with mask | 127.0.0.1\|255.255.255.0 | + | CIDR IP address | 127.0.0.1⁄24 | + | IP address segment | 127.0.0.1-127.0.0.5 | + +## Dependencies + +In the public cloud service scenario, the OSS or OBS is required for log storage. \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/database-security/9-dynamic-data-anonymization.md b/product/en/docs-mogdb/v2.1/characteristic-description/database-security/9-dynamic-data-anonymization.md new file mode 100644 index 0000000000000000000000000000000000000000..1a4dc10a255bb3bcfc0df8c0dab3db393a40c09c --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/database-security/9-dynamic-data-anonymization.md @@ -0,0 +1,106 @@ +--- +title: Dynamic Data Masking +summary: Dynamic Data Masking +author: Guo Huan +date: 2022-05-09 +--- + +# Dynamic Data Masking + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +Data masking is an effective database privacy protection solution, which can prevent attackers from snooping on private data. The dynamic data masking mechanism is a technology that protects privacy data by customizing masking policies. It can effectively prevent unauthorized users from accessing sensitive information while retaining original data. After the administrator specifies the object to be anonymized and customizes a data masking policy, if the database resources queried by a user are associated with a masking policy, data is anonymized based on the user identity and masking policy to restrict attackers' access to privacy data. + +## Benefits + +Data privacy protection is one of the required database security capabilities. It can restrict attackers' access to privacy data, ensuring privacy data security. The dynamic data masking mechanism can protect the privacy of specified database resources by configuring masking policies. In addition, the masking policy configuration is flexible and can implement targeted privacy protection in specific user scenarios. + +## Description + +The dynamic data masking mechanism customizes masking policies based on resource labels. It can select masking modes based on the site requirements or customize masking policies for specific users. The SQL syntax for creating a complete masking policy is as follows: + +``` +CREATE RESOURCE LABEL label_for_creditcard ADD COLUMN(user1.table1.creditcard); +CREATE RESOURCE LABEL label_for_name ADD COLUMN(user1.table1.name); +CREATE MASKING POLICY msk_creditcard creditcardmasking ON LABEL(label_for_creditcard); +CREATE MASKING POLICY msk_name randommasking ON LABEL(label_for_name) FILTER ON IP(local), ROLES(dev); +``` + +**label_for_creditcard** and **msk_name** are the resource labels for masking, and each label is allocated to two column objects. **creditcardmasking** and **randommasking** are preset masking functions. **msk_creditcard** specifies that the masking policy **creditcardmasking** will be applied when any user accesses resources with **label_for_creditcard**, regardless of the access source. **msk_name** specifies that the masking policy **randommasking** will be applied when local user **dev** accesses resources with **label_for_name**. If **FILTER** is not specified, the setting takes effect for all users. Otherwise, the setting takes effect only for specified users. + +The following table shows the preset masking functions: + +| **Masking Function** | **Example** | +| -------------------- | ------------------------------------------------------------ | +| creditcardmasking | '4880-9898-4545-2525' will be anonymized as 'xxxx-xxxx-xxxx-2525'. This function anonymizes digits except the last four digits. | +| basicemailmasking | 'abcd@gmail.com' will be anonymized as 'xxxx@gmail.com'. This function anonymizes text before the first @. | +| fullemailmasking | 'abcd@gmail.com' will be anonymized as 'xxxx@xxxxx.com'. This function anonymizes text before the first dot (.) (except @). | +| alldigitsmasking | 'alex123alex' will be anonymized as 'alex000alex'. This function anonymizes only digits in the text. | +| shufflemasking | 'hello word' will be anonymized as 'hlwoeor dl'. This weak masking function is implemented through character dislocation. You are not advised to use this function to anonymize strings with strong semantics. | +| randommasking | 'hello word' will be anonymized as 'ad5f5ghdf5'. This function randomly anonymizes text by character. | +| maskall | '4880-9898-4545-2525' will be anonymized as 'xxxxxxxxxxxxxxxxxxx'. | + +The data types supported by each masking function are as follows: + +| **Masking Function** | **Supported Data Types** | +| -------------------- | ------------------------------------------------------------ | +| creditcardmasking | BPCHAR, VARCHAR, NVARCHAR, TEXT (character data in credit card format only) | +| basicemailmasking | BPCHAR, VARCHAR, NVARCHAR, TEXT (character data in email format only) | +| fullemailmasking | BPCHAR, VARCHAR, NVARCHAR, TEXT (character data in email format only) | +| alldigitsmasking | BPCHAR, VARCHAR, NVARCHAR, TEXT (character data containing digits only) | +| shufflemasking | BPCHAR, VARCHAR, NVARCHAR, TEXT (text data only) | +| randommasking | BPCHAR, VARCHAR, NVARCHAR, TEXT (text data only) | +| maskall | BOOL, RELTIME, TIME, TIMETZ, INTERVAL, TIMESTAMP, TIMESTAMPTZ, SMALLDATETIME, ABSTIME,TEXT, BPCHAR, VARCHAR, NVARCHAR2, NAME, INT8, INT4, INT2, INT1, NUMRIC, FLOAT4, FLOAT8, CASH | + +For unsupported data types, the **maskall** function is used for data masking by default. The data of the BOOL type is masked as **'0'**. The RELTIME type is masked as **'1970'**. The TIME, TIMETZ, and INTERVAL types are masked as **'00:00:00.0000+00'**. The TIMESTAMP, TIMESTAMPTZ, SMALLDATETIME, and ABSTIME types are masked as **'1970-01-01 00:00:00.0000'**. The TEXT, CHAR, BPCHAR, VARCHAR, NVARCHAR2, and NAME type are masked as **'x'**. The INT8, INT4, INT2, INT1, NUMERIC, FLOAT4, FLOAT8 types are masked as **'0'**. If the data type is not supported by **maskall**, the masking policy cannot be created. If implicit conversion is involved in the masking column, the data type after implicit conversion is used for masking. In addition, if the masking policy is applied to a data column and takes effect, operations on the data in the column are performed based on the masking result. + +Dynamic data masking applies to scenarios closely related to actual services. It provides users with proper masking query APIs and error handling logic based on service requirements to prevent raw data from being obtained through credential stuffing. + +## Enhancements + +None. + +## Constraints + +- The dynamic data masking policy must be created by a user with the **POLADMIN** or **SYSADMIN** attribute, or by the initial user. Common users do not have the permission to access the security policy system catalog and system view. + +- Dynamic data masking takes effect only on data tables for which masking policies are configured. Audit logs are not within the effective scope of the masking policies. + +- In a masking policy, only one masking mode can be specified for a resource label. + +- Multiple masking policies cannot be used to anonymize the same resource label, except when **FILTER** is used to specify user scenarios where the policies take effect and there is no intersection between user scenarios of different masking policies that contain the same resource label. In this case, you can identify the policy that a resource label is anonymized by based on the user scenario. + +- It is recommended that **APP** in **FILTER** be set to applications in the same trusted domain. Since a client may be forged, a security mechanism must be formed on the client when **APP** is used to reduce misuse risks. Generally, you are not advised to set **APP**. If it is set, pay attention to the risk of client spoofing. + +- For INSERT or MERGE INTO operations with the query clause, if the source table contains anonymized columns, the inserted or updated result in the preceding two operations is the anonymized value and cannot be restored. + +- When the built-in security policy is enabled, the ALTER TABLE EXCHANGE PARTITION statement fails to be executed if the source table is in the anonymized column. + +- If a dynamic data masking policy is configured for a table, grant the trigger permission of the table to other users with caution to prevent other users from using the trigger to bypass the masking policy. + +- A maximum of 98 dynamic data masking policies can be created. + +- Only the preceding seven preset masking policies can be used. + +- Only data with the resource labels containing the **COLUMN** attribute can be anonymized. + +- Only columns in base tables can be anonymized. + +- Only the data queried using **SELECT** can be anonymized. + +- Taking an IPv4 address as an example, the following formats are supported: + + | IP Address Format | Example | + | -------------------- | ------------------------ | + | Single IP address | 127.0.0.1 | + | IP address with mask | 127.0.0.1\|255.255.255.0 | + | CIDR IP address | 127.0.0.1⁄24 | + | IP address segment | 127.0.0.1-127.0.0.5 | + +## Dependencies + +None. \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/1-support-for-functions-and-stored-procedures.md b/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/1-support-for-functions-and-stored-procedures.md new file mode 100644 index 0000000000000000000000000000000000000000..a92e94fe5cc6eeb9f6a93eacdccc8ff4516bf4ac --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/1-support-for-functions-and-stored-procedures.md @@ -0,0 +1,38 @@ +--- +title: Support for Functions and Stored Procedures +summary: Support for Functions and Stored Procedures +author: Guo Huan +date: 2022-05-09 +--- + +# Support for Functions and Stored Procedures + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +Functions and stored procedures are important database objects. They encapsulate SQL statement sets used for certain functions so that the statements can be easily invoked. + +## Benefits + +1. Allows customers to modularize program design and encapsulate SQL statement sets, easy to invoke. +2. Caches the compilation results of stored procedures to accelerate SQL statement set execution. +3. Allows system administrators to restrict the permission for executing a specific stored procedure and controls access to the corresponding type of data. This prevents access from unauthorized users and ensures data security. + +## Description + +MogDB supports functions and stored procedures compliant with the SQL standard. The stored procedures are compatible with certain mainstream stored procedure syntax, improving their usability. + +## Enhancements + +PL/pgSQL single-step debugging is supported. + +## Constraints + +None. + +## Dependencies + +None. \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/10-autonomous-transaction.md b/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/10-autonomous-transaction.md new file mode 100644 index 0000000000000000000000000000000000000000..f0be83458d533b2a6b4e08da7258af3f067b6286 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/10-autonomous-transaction.md @@ -0,0 +1,44 @@ +--- +title: Autonomous Transaction +summary: Autonomous Transaction +author: Guo Huan +date: 2022-05-09 +--- + +# Autonomous Transaction + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +An autonomous transaction is a type of transaction in which the commit of a sub-transaction is not affected by the commit or rollback of the main transaction. + +## Benefits + +This feature meets diversified application scenarios. + +## Description + +In an autonomous transaction, a specified type of SQL statements are executed in an independent transaction context during the execution of the main transaction. The commit and rollback operations of an autonomous transaction are not affected by the commit and rollback operations of the main transaction. + +User-defined functions and stored procedures support autonomous transactions. + +A typical application scenario is as follows: A table is used to record the operation information during the main transaction execution. When the main transaction fails to be rolled back, the operation information recorded in the table cannot be rolled back. + +## Enhancements + +None + +## Constraints + +- A trigger function does not support autonomous transactions. +- In the autonomous transaction block of a function or stored procedure, static SQL statements do not support variable transfer. +- Autonomous transactions do not support nesting. +- A function containing an autonomous transaction does not support the return value of parameter transfer. +- A stored procedure or function that contains an autonomous transaction does not support exception handling. + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/11-global-temporary-table.md b/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/11-global-temporary-table.md new file mode 100644 index 0000000000000000000000000000000000000000..d64dfc4c1693b3364f7e975729b616adaec51e18 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/11-global-temporary-table.md @@ -0,0 +1,45 @@ +--- +title: Global Temporary Table +summary: Global Temporary Table +author: Guo Huan +date: 2022-05-09 +--- + +# Global Temporary Table + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +A temporary table does not guarantee persistency. Its life cycle is usually bound to a session or transaction, which can be used to store temporary data during processing and accelerate query. + +## Benefits + +This feature improves the expression capability and usability of temporary tables. + +## Description + +The metadata of the global temporary table is visible to all sessions. After the sessions end, the metadata still exists. The user data, indexes, and statistics of a session are isolated from those of another session. Each session can only view and modify the data submitted by itself. + +Global temporary tables have two schemas: ON COMMIT PRESERVE ROWS and ON COMMIT PRESERVE ROWS. In session-based ON COMMIT PRESERVE ROWS schema, user data is automatically cleared when a session ends. In transaction-based ON COMMIT DELETE ROWS schema, user data is automatically cleared when the commit or rollback operation is performed. If the **ON COMMIT** option is not specified during table creation, the session level is used by default. Different from local temporary tables, you can specify a schema that does not start with **pg_temp_** when creating a global temporary table. + +## Enhancements + +The processing of the global temporary table is added based on the local temporary table. + +## Constraints + +- Parallel scanning is not supported. +- Temp tablespace is not supported. +- Partitions are not supported. +- GIST indexes are not supported. +- The user-defined statistics **pg_statistic_ext** is not supported. +- ON COMMIT DROP is not supported. +- Hash bucket cluster storage is not supported. +- Row store is not supported. + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/12-pseudocolumn-rownum.md b/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/12-pseudocolumn-rownum.md new file mode 100644 index 0000000000000000000000000000000000000000..12817fca6df7ed9c756042a0a11d81cde945727d --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/12-pseudocolumn-rownum.md @@ -0,0 +1,43 @@ +--- +title: Pseudocolumn ROWNUM +summary: Pseudocolumn ROWNUM +author: Guo Huan +date: 2022-05-09 +--- + +# Pseudocolumn ROWNUM + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +ROWNUM is a sequence number generated for each record in the query result. The sequence number starts from 1 and is unique. + +## Benefits + +- This feature is compatible with Oracle features, facilitating database migration. +- Similar to the LIMIT feature, this feature can filter out the first *n* records in the result set. + +## Description + +ROWNUM (pseudocolumn), which is used to label the records that meet conditions in the SQL query in sequence. In the query result, the value of **ROWNUM** in the first line is **1**, the value of **ROWNUM** in the second line is **2**, and so on. The value of **ROWNUM** in the _n_th line is *n*. This feature is used to filter the first *n* rows of data in the query result set, which is similar to the LIMIT function in MogDB. + +## Enhancements + +During internal execution, the optimizer rewrites ROWNUM into LIMIT to accelerate the execution speed. + +## Constraints + +- Do not use the pseudocolumn ROWNUM as an alias to avoid ambiguity in SQL statements. +- Do not use ROWNUM when creating an index. Bad example: **create index index_name on table(rownum);** +- Do not use ROWNUM as the default value when creating a table. Bad example: **create table table_name(id int default rownum);** +- Do not use ROWNUM as an alias in the WHERE clause. Bad example: **select rownum rn from table where rn < 5;** +- Do not use ROWNUM when inserting data. Bad example: **insert into table values (rownum,'blue')** +- Do not use ROWNUM in a table-less query. Bad example: **select \* from (values(rownum,1)), x(a,b);** +- If the HAVING clause contains ROWNUM (and is not in the aggregate function), the GROUP BY clause must contain ROWNUM (and is not in the aggregate function). + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/13-stored-procedure-debugging.md b/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/13-stored-procedure-debugging.md new file mode 100644 index 0000000000000000000000000000000000000000..ba6e854074bf6a0eb24bbac724090d42d64dc2a0 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/13-stored-procedure-debugging.md @@ -0,0 +1,38 @@ +--- +title: Stored Procedure Debugging +summary: Stored Procedure Debugging +author: Guo Huan +date: 2022-05-09 +--- + +# Stored Procedure Debugging + +## Availability + +This feature was introduced in MogDB 1.1.0. After the third-party database code directory structure was adjusted, this feature was temporarily deleted and is now available since MogDB 1.1.0. + +## Introduction + +This feature provides a group of APIs for debugging stored procedures, such as breakpoint debugging and variable printing. + +## Benefits + +This feature improves user experience in developing stored procedures based on MogDB. + +## Description + +Stored procedures are important database objects. They encapsulate SQL statement sets used for certain functions so that the statements can be easily invoked. A stored procedure usually contains many SQL statements and procedural execution structures, depending on the service scale. However, writing a large stored procedure is usually accompanied by logic bugs. It is difficult or even impossible to find the bugs by only executing the stored procedure. Therefore, a debugging tool is required. + +The stored procedure debugging tool provides a group of debugging APIs to enable the stored procedure to be executed step by step. During the execution, you can set breakpoints and print variables so that SQL developers can detect and correct errors in time and develop functions more efficiently and with high quality. + +## Enhancements + +None + +## Constraints + +None + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/14-jdbc-client-load-balancing-and-readwrite-isolation.md b/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/14-jdbc-client-load-balancing-and-readwrite-isolation.md new file mode 100644 index 0000000000000000000000000000000000000000..ef67d177d90a9ca94c58a89f2ef6ec87f818b4a2 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/14-jdbc-client-load-balancing-and-readwrite-isolation.md @@ -0,0 +1,36 @@ +--- +title: JDBC Client Load Balancing and Read/Write Isolation +summary: JDBC Client Load Balancing and Read/Write Isolation +author: Guo Huan +date: 2022-05-09 +--- + +# JDBC Client Load Balancing and Read/Write Isolation + +## Availability + +This feature is available since MogDB 2.1.0. + +## Introduction + +The JDBC client provides load balancing and read/write isolation capabilities. + +## Benefits + +Load balancing and read/write isolation can be configured on the JDBC client. + +## Description + +The IP addresses and port numbers of multiple nodes on the client are configured to adapt to HA switchover between multiple AZs and remote DR switchover. The connection-level read/write isolation configuration is supported. Preferentially connecting to read-only nodes is supported. Multiple read-only nodes are evenly distributed. + +## Enhancements + +None. + +## Constraints + +None. + +## Dependencies + +None. \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/15-in-place-update-storage-engine.md b/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/15-in-place-update-storage-engine.md new file mode 100644 index 0000000000000000000000000000000000000000..200950ee678e127d7a33185c423061f5d0667e73 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/15-in-place-update-storage-engine.md @@ -0,0 +1,36 @@ +--- +title: In-place Update Storage Engine +summary: In-place Update Storage Engine +author: Guo Huan +date: 2022-05-09 +--- + +# In-place Update Storage Engine + +## Availability + +This feature is available since MogDB 2.1.0. + +## Introduction + +The in-place update storage engine is a new storage mode added to the MogDB. The row storage engine used by the earlier MogDB versions is in append update mode. The append update has good performance in addition, deletion, and HOT (Heap Only Tuple) update (that is, update on the same page) in the service. However, in a non-HOT UPDATE scenario across data pages, garbage collection is not efficient. The Ustore storage engine can solve this problem. + +## Benefits + +The in-place update storage engine can effectively reduce storage space occupation after tuples are updated for multiple times. + +## Description + +The in-place update storage engine solves the problems of space expansion and large tuples of the Append update storage engine. The design of efficient rollback segments is the basis of the in-place update storage engine. + +## Enhancements + +None. + +## Constraints + +None. + +## Dependencies + +None. \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/2-sql-hints.md b/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/2-sql-hints.md new file mode 100644 index 0000000000000000000000000000000000000000..8ad224336c7d788fafadf279e83c8b33266bbbb9 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/2-sql-hints.md @@ -0,0 +1,40 @@ +--- +title: SQL Hints +summary: SQL Hints +author: Guo Huan +date: 2022-05-09 +--- + +# SQL Hints + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +SQL hints can be used to override execution plans. + +## Benefits + +Improves SQL query performance. + +## Description + +In plan hints, you can specify a join order; join, stream, and scan operations, the number of rows in a result, and redistribution skew information to tune an execution plan, improving query performance. + +## Enhancements + +The session-level optimizer parameters can be set by using planhint. + +The specified subquery can be not expanded. + +GPC can be disabled for a single query. + +## Constraints + +None. + +## Dependencies + +None. \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/3-full-text-indexing.md b/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/3-full-text-indexing.md new file mode 100644 index 0000000000000000000000000000000000000000..101c78b7b9135ed88d326f0f1eb838dabcc6c75b --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/3-full-text-indexing.md @@ -0,0 +1,53 @@ +--- +title: Full-Text Indexing +summary: Full-Text Indexing +author: Guo Huan +date: 2022-05-09 +--- + +# Full-Text Indexing + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +MogDB full-text indexing allows documents to be preprocessed and facilitates subsequent search. + +## Benefits + +MogDB full-text indexing provides the capability to identify natural-language documents that satisfy a query and sort them by relevance. + +## Description + +The preprocessing process of creating a full-text index includes: + +- Parsing documents into tokens + + It is useful to identify various classes of tokens, for example, numbers, words, compound words, and email addresses, so that they can be processed differently. In principle, token classes depend on the specific application, but for most purposes it is adequate to use a predefined set of classes. + +- Converting tokens into lexemes + + A lexeme is a string, just like a token, but it has been normalized so that different forms of the same word are made alike. For example, normalization almost always includes folding upper-case letters to lower-case, and often involves removal of suffixes (such as **s** or **es** in English). This allows searches to find variant forms of the same word, without entering all the possible variants. Also, this step typically eliminates stop words, which are so common and usually useless for searching. (In short, tokens are raw fragments of the document text, while lexemes are words that are believed useful for indexing and searching.) MogDB uses dictionaries to perform this step and provides various standard dictionaries. + +- Storing preprocessed documents optimized for searching + + For example, each document can be represented as a sorted array of normalized lexemes. Along with the lexemes, it is often desirable to store positional information for proximity ranking. Therefore, a document that contains a more “dense” area of query words is assigned with a higher rank than the one with scattered query words. Dictionaries allow fine-grained control over how tokens are normalized. With appropriate dictionaries, you can define stop words that should not be indexed. + +## Enhancements + +None + +## Constraints + +The current limitations of MogDB's text search features are: + +- The length of each lexeme must be less than 2 KB. +- The length of a **tsvector** (lexemes + positions) must be less than 1 MB. +- Position values in **tsvector** must be greater than 0 and less than or equal to 16383. +- No more than 256 positions per lexeme. Excessive positions, if any, will be discarded. + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/4-copy-interface-for-error-tolerance.md b/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/4-copy-interface-for-error-tolerance.md new file mode 100644 index 0000000000000000000000000000000000000000..674c22f8659962162f3b72f1e6f90fe524493ebc --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/4-copy-interface-for-error-tolerance.md @@ -0,0 +1,36 @@ +--- +title: Copy Interface for Error Tolerance +summary: Copy Interface for Error Tolerance +author: Guo Huan +date: 2022-05-09 +--- + +# Copy Interface for Error Tolerance + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +Certain errors that occur during the copy process are imported to a specified error table without interrupting the process. + +## Benefits + +Refine the copy function and improve the tolerance and robustness to common errors such as invalid formats. + +## Description + +MogDB provides the encapsulated copy error tables for creating functions and allows users to specify error tolerance options when using the **Copy From** statement. In this way, errors related to parsing, data format, and character set during the execution of the **Copy From** statement are recorded in the error table instead of being reported and interrupted. Even if a small amount of data in the target file of **Copy From** is incorrect, the data can be imported to the database. You can locate and rectify the fault in the error table later. + +## Enhancements + +None + +## Constraints + +For details, see “Importing Data > Running the COPY FROM STDIN Statement to Import Data > [Handling Import Errors](3-running-the-COPY-FROM-STDIN-statement-to-import-data#handling-import-errors)” in the *Administrator Guide*. + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/5-partitioning.md b/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/5-partitioning.md new file mode 100644 index 0000000000000000000000000000000000000000..110804c072ab82fcf18565649560305dc372ed65 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/5-partitioning.md @@ -0,0 +1,58 @@ +--- +title: Partitioning +summary: Partitioning +author: Guo Huan +date: 2022-05-09 +--- + +# Partitioning + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +Data is partitioned horizontally on a node using a specified policy. This operation splits a table into multiple partitions that are not overlapped. + +## Benefits + +In common scenarios, a partitioned table has the following advantages over a common table: + +- High query performance: You can specify partitions when querying partitioned tables, improving query efficiency. +- High availability: If a certain partition in a partitioned table is faulty, data in the other partitions is still available. +- Balanced I/O: Partitions can be mapped to different disks to balance I/O and improve the overall system performance. + +## Description + +Currently, MogDB supports range partitioned tables, list partitioned tables, and hash partitioned tables. + +- In a range partitioned table, data within a certain range is mapped to each partition. The range is determined by the partition key specified when the partitioned table is created. This partitioning mode is most commonly used. + + With the range partitioning function, the database divides a record, which is to be inserted into a table, into multiple ranges using one or multiple columns and creates a partition for each range to store data. Partition ranges do no overlap. + +- In a list partitioned table, data is mapped to each partition based on the key values contained in each partition. The key values contained in a partition are specified when the partition is created. + + The list partitioning function divides the key values in the records to be inserted into a table into multiple lists (the lists do not overlap in different partitions) based on a column of the table, and then creates a partition for each list to store the corresponding data. + +- In a hash partitioned table, data is mapped to each partition using the hash algorithm, and each partition stores records with the same hash value. + + The hash partitioning function uses the internal hash algorithm to divide records to be inserted into a table into partitions based on a column of the table. + +If you specify the **PARTITION** parameter when running the **CREATE TABLE** statement, data in the table will be partitioned. Users can modify partition keys as needed during table creation to make the query result stored in the same or least partitions (called partition pruning), obtaining consecutive I/O to improve the query performance. + +In actual services, time is often used to filter query objects. Therefore, you can select the time column as the partition key. The key value range can be adjusted based on the total data volume and the data volume queried at a time. + +## Enhancements + +Range partitioned tables can be combined. + +The list and hash partitioned tables support data exchange. + +## Constraints + +None. + +## Dependencies + +None. \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/6-support-for-advanced-analysis-functions.md b/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/6-support-for-advanced-analysis-functions.md new file mode 100644 index 0000000000000000000000000000000000000000..59ca49d46dc9fdef6dc701c7ee26bc46f89c3a84 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/6-support-for-advanced-analysis-functions.md @@ -0,0 +1,57 @@ +--- +title: Support for Advanced Analysis Functions +summary: Support for Advanced Analysis Functions +author: Guo Huan +date: 2022-05-09 +--- + +# Support for Advanced Analysis Functions + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +None + +## Benefits + +Window functions are provided for advanced data analysis and processing. The window function groups the data in a table in advance. Each row belongs to a specific group. Then, a series of association analysis calculations are performed on the group. In this way, some attributes of each tuple in the set and association information with other tuples can be mined. + +## Description + +The following uses an example to describe the window analysis function: Compare the salary of each person in a department with the average salary of the department. + +``` +SELECT depname, empno, salary, avg(salary) OVER (PARTITION BY depname) FROM empsalary; +depname | empno | salary | avg +-----------+-------+--------+----------------------- +develop | 11 | 5200 | 5020.0000000000000000 +develop | 7 | 4200 | 5020.0000000000000000 +develop | 9 | 4500 | 5020.0000000000000000 +develop | 8 | 6000 | 5020.0000000000000000 +develop | 10 | 5200 | 5020.0000000000000000 +personnel | 5 | 3500 | 3700.0000000000000000 +personnel | 2 | 3900 | 3700.0000000000000000 +sales | 3 | 4800 | 4866.6666666666666667 +sales | 1 | 5000 | 4866.6666666666666667 +sales | 4 | 4800 | 4866.6666666666666667 +(10 rows) +``` + +The analysis function **avg(salary) OVER (PARTITION BY depname)** easily calculates each employee's salary and the average salary of the department. + +Currently, the system supports the following analysis functions: **row_number()**, **rank()**, **dense_rank()**, **percent_rank()**, **cume_dist()**, **ntile()**, **lag()**, **lead()**, **first_value()**, **last_value()**, and **nth_value()**. For details about functions and statements, see “Functions and Operators > [Window Functions](18-window-functions)” in the *Reference Guide*. + +## Enhancements + +None + +## Constraints + +None + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/7-materialized-view.md b/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/7-materialized-view.md new file mode 100644 index 0000000000000000000000000000000000000000..596d4c1218c31f7c0ca12c97c9edb6ae91479354 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/7-materialized-view.md @@ -0,0 +1,36 @@ +--- +title: Materialized View +summary: Materialized View +author: Guo Huan +date: 2022-05-09 +--- + +# Materialized View + +## Availability + +This feature is available since V500R001C10. + +## Introduction + +A materialized view is a special physical table, which is relative to a common view. A common view is a virtual table and has many application limitations. Any query on a view is actually converted into a query on an SQL statement, and performance is not actually improved. The materialized view actually stores the results of the statements executed by the SQL statement, and is used to cache the results. + +## Benefits + +The materialized view function is used to improve query efficiency. + +## Description + +Full materialized views and incremental materialized views are supported. Full materialized views can only be updated in full mode. Incremental materialized views can be updated asynchronously. You can run statements to update new data to materialized views. + +## Enhancements + +None + +## Constraints + +Only simple filter queries and UNION ALL statements are supported for base tables. + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/8-hyperloglog.md b/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/8-hyperloglog.md new file mode 100644 index 0000000000000000000000000000000000000000..1e23d5c4a5c5e1cfe42499018b1fa4cc2be58c43 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/8-hyperloglog.md @@ -0,0 +1,38 @@ +--- +title: HyperLogLog +summary: HyperLogLog +author: Guo Huan +date: 2022-05-09 +--- + +# HyperLogLog + +## Availability + +This feature is available as of MogDB 1.1.0. + +## Introduction + +HyperLoglog (HLL) is used to count the number of distinct values. + +## Benefits + +Improves AP/TP query performance. + +## Description + +HLL is an approximation algorithm for efficiently counting the number of distinct values in a dataset. It features faster computing and lower space usage. You only need to store HLL data structures instead of datasets. When new data is added to a dataset, make hash calculation on the data and insert the result to an HLL. Then, you can obtain the final result based on the HLL. + +HLL has advantages over others in the computing speed and storage space requirement. In terms of time complexity, the Sort algorithm needs to sort at least O(n log n) time. Although the Hash algorithm can obtain the result by scanning the entire table O(n) time, the storage space is as follows: Both the Sort and Hash algorithms need to store the original data before collecting statistics, which consumes a large amount of storage space. For the HLL, the original data does not need to be stored, and only the HLL data structure needs to be maintained. Therefore, the occupied space is always at the 1280-byte constant level. + +## Enhancements + +None + +## Constraints + +None + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/9-creating-an-index-online.md b/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/9-creating-an-index-online.md new file mode 100644 index 0000000000000000000000000000000000000000..1ee96865908d3e0f5b6a28e6b6843c9477ffad58 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/9-creating-an-index-online.md @@ -0,0 +1,40 @@ +--- +title: Creating an Index Online +summary: Creating an Index Online +author: Guo Huan +date: 2022-05-09 +--- + +# Creating an Index Online + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +Uses the CREATE INDEX CONCURRENTLY syntax to create indexes online without blocking DML. + +## Benefits + +When creating an index, you can specify the CONCURRENTLY keyword to ensure that the DML and online services are not blocked during the index creation. + +## Description + +A normal CREATE INDEX acquires exclusive lock on the table on which the index depends, blocking other accesses until the index drop can be completed. If the CONCURRENTLY keyword is specified, the ShareUpdateExclusiveLock lock is added to the table so that DML is not blocked during the creation. + +This keyword is specified when an index is created online. The entire table needs to be scanned twice and built. When the table is scanned for the first time, an index is created and the read and write operations are not blocked. During the second scan, changes that have occurred since the first scan are merged and updated. The table needs to be scanned and built twice, and all existing transactions that may modify the table must be completed. This means that the creation of the index takes a longer time than normal. In addition, the CPU and I/O consumption also affects other services. + +## Enhancements + +None + +## Constraints + +- Only one index name can be specified when an index is created online. +- The CREATE INDEX statement can be run within a transaction, but CREATE INDEX CONCURRENTLY cannot. +- Column-store tables, partitioned tables, and temporary tables do not support CREATE INDEX CONCURRENTLY. + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/high-availability/1-primary-standby.md b/product/en/docs-mogdb/v2.1/characteristic-description/high-availability/1-primary-standby.md new file mode 100644 index 0000000000000000000000000000000000000000..f0500a60bca41e174ecb8d3693db86e5b9a07320 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/high-availability/1-primary-standby.md @@ -0,0 +1,42 @@ +--- +title: Primary/Standby +summary: Primary/Standby +author: Guo Huan +date: 2022-05-09 +--- + +# Primary/Standby + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +To ensure that a fault can be rectified, data needs to be written into multiple copies. Multiple copies are configured for the primary and standby nodes, and logs are used for data synchronization. In this way, MogDB has no data lost when a node is faulty or the system restarts after a stop, meeting the ACID feature requirements. + +## Benefits + +Services can be switched to the standby node when the primary node is faulty. Therefore, data is not lost and services can be quickly restored. + +## Description + +The primary/standby environment supports two modes: primary/standby/secondary and one-primary-multiple-standby. In the primary/standby/secondary mode, the standby node needs to redo logs and can be promoted to the primary. However, the secondary node can only receive logs and cannot be promoted to the primary. In the one-primary-multiple-standby mode, all standby nodes need to redo logs and can be promoted to the primary. The primary/standby/secondary mode is mainly used for the OLAP system, saving storage resources. The one-primary-multiple-standby mode provides higher DR capabilities and is more suitable for the OLTP system that processes a large number of transactions. + +The **switchover** command can be used to trigger a switchover between the primary and standby nodes. If the primary node is faulty, the **failover** command can be used to promote the standby node to the primary. + +In scenarios such as initial installation or backup and restoration, data on the standby node needs to be rebuilt based on the primary node. In this case, the build function is required to send the data and WALs of the primary node to the standby node. When the primary node is faulty and joins again as a standby node, the build function needs to be used to synchronize data and WALs with those of the new primary node. In addition, in online capacity expansion scenarios, you need to use build to synchronize metadata to instances on new nodes. Build includes full build and incremental build. Full build depends on primary node data for rebuild. The amount of data to be copied is large and the time required is long. Incremental build copies only differential files. The amount of data to be copied is small and the time required is short. Generally, the incremental build is preferred for fault recovery. If the incremental build fails, the full build continues until the fault is rectified. + +To implement HA DR for all instances, in addition to the preceding primary/standby multi-copy replication configured for DNs, MogDB also provides other primary/standby DR capabilities, such as CM server (one primary and multiple standbys) and ETCD (one primary and multiple standbys). In this way, instances can be recovered as soon as possible without interrupting services, minimizing the impact of hardware, software, and human errors on services and ensuring service continuity. + +## Enhancements + +None + +## Constraints + +None + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/high-availability/10-adding-or-deleting-a-standby-server.md b/product/en/docs-mogdb/v2.1/characteristic-description/high-availability/10-adding-or-deleting-a-standby-server.md new file mode 100644 index 0000000000000000000000000000000000000000..0aac767cfda871a806af9ccaeb35fc18a24eb156 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/high-availability/10-adding-or-deleting-a-standby-server.md @@ -0,0 +1,58 @@ +--- +title: Adding or Deleting a Standby Server +summary: Adding or Deleting a Standby Server +author: Guo Huan +date: 2022-05-09 +--- + +# Adding or Deleting a Standby Server + +## Availability + +This feature is available since MogDB 2.0.0. + +## Introduction + +Standby servers can be added and deleted. + +## Benefits + +If the read pressure of the primary server is high or you want to improve the disaster recovery capability of the database, you need to add a standby server. If some standby nodes in a cluster are faulty and cannot be recovered within a short period of time, you can delete the faulty nodes to ensure that the cluster is running properly. + +## Description + +MogDB can be scaled out from a single server or one primary and multiple standbys to one primary and eight standbys. Cascaded standby servers can be added. Standby nodes can be added when a faulty standby server exists in the cluster. One primary and multiple standbys can be scaled in to a single server. A faulty standby server can be deleted. + +Standby nodes can be added or deleted online without affecting the primary server. + +## Enhancements + +None + +## Constraints + +For adding a standby server: + +- Ensure that the MogDB image package exists on the primary server. +- Ensure that the same users and user groups as those on the primary server have been created on the new standby server. +- Ensure that the mutual trust of user **root** and the database management user has been established between the existing database nodes and the new nodes. +- Ensure that the XML file has been properly configured and information about the standby server to be scaled has been added to the installed database configuration file. +- Ensure that only user **root** is authorized to run the scale-out command. +- Do not run the **gs_dropnode** command on the primary server to delete other standby nodes at the same time. +- Ensure that the environment variables of the primary server have been imported before the scale-out command is run. +- Ensure that the operating system of the new standby server is the same as that of the primary server. +- Do not perform an primary/standby switchover or failover on other standby nodes at the same time. + +For deleting a standby server: + +- Delete the standby node only on the primary node. +- Do not perform an primary/standby switchover or failover on other standby nodes at the same time. +- Do not run the **gs_expansion** command on the primary node for scale-out at the same time. +- Do not run the **gs_dropnode** command twice at the same time. +- Before deletion, ensure that the database management user trust relationship has been established between the primary and standby nodes. +- Run this command as a database administrator. +- Before running commands, run the **source** command to import environment variables of the primary server. + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/high-availability/2-logical-replication.md b/product/en/docs-mogdb/v2.1/characteristic-description/high-availability/2-logical-replication.md new file mode 100644 index 0000000000000000000000000000000000000000..3c092d8ec5eaef0429d959e673f9c0c7baa76195 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/high-availability/2-logical-replication.md @@ -0,0 +1,37 @@ +--- +title: Logical Replication +summary: Logical Replication +author: Guo Huan +date: 2022-05-09 +--- + +# Logical Replication + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +MogDB provides the logical decoding function to reversely parse physical logs to logical logs. Logical replication tools such as DRS convert logical logs to SQL statements and replay the SQL statements in the peer database. In this way, data can be synchronized between heterogeneous databases. Currently, unidirectional and bidirectional logical replication between the MogDB database and the MySQL or Oracle database is supported. + +## Benefits + +Logical replication is applicable to real-time database data migration, dual-database active-active system, and rolling upgrades. + +## Description + +DNs reversely parse physical logs to logical logs. Logical replication tools such as DRS extract logical logs from DNs, convert the logs to SQL statements, and replay the SQL statements in MySQL. Logical replication tools also extract logical logs from a MySQL database, reversely parse the logs to SQL statements, and replay the SQL statements in MogDB. In this way, data can be synchronized between heterogeneous databases. + +## Enhancements + +- MogDB 1.1.0 logic decoding supports the extraction of logs from full and incremental logs. +- MogDB 1.1.0 supports logical decoding on a standby node. + +## Constraints + +Column-store replication and DDL replication are not supported. + +## Dependencies + +It depends on logical replication tools that decode logical logs. \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/high-availability/3-online-node-replacement.md b/product/en/docs-mogdb/v2.1/characteristic-description/high-availability/3-online-node-replacement.md new file mode 100644 index 0000000000000000000000000000000000000000..d506d92914f8f642d2012694f0533ecc9cd24726 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/high-availability/3-online-node-replacement.md @@ -0,0 +1,38 @@ +--- +title: Online Node Replacement +summary: Online Node Replacement +author: Guo Huan +date: 2022-05-09 +--- + +# Online Node Replacement + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +If a node in a database is unavailable or the instance status is abnormal due to a hardware fault and the database is not locked, you can replace the node or rectify the instance fault to restore the database. During the restoration, DML operations are supported. DDL operations are supported in limited scenarios only. + +## Benefits + +Currently, the scale of enterprise data is increasing, the number of nodes increases sharply, and the probability of hardware damage increases accordingly. The traditional offline node replacement mode cannot meet customer requirements for uninterrupted services. During routine O&M, frequent service interruption will bring great loss to customers. However, the current database products in the industry cannot meet the requirements of physical node replacement in large-scale data scenarios without service interruption. Services need to be interrupted, or only some operations are allowed when services are not interrupted. + +## Description + +If a node in a database is unavailable or the instance status is abnormal due to a hardware fault, you can replace the node or rectify the instance fault to restore the database. During the restoration, DML operations are supported. DDL operations are supported in limited scenarios only. + +## Enhancements + +None + +## Constraints + +Currently, online DDL operations are supported during node replacement. + +- During node replacement, DML operations are supported and DDL operations are supported in certain scenarios. + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/high-availability/4-logical-backup.md b/product/en/docs-mogdb/v2.1/characteristic-description/high-availability/4-logical-backup.md new file mode 100644 index 0000000000000000000000000000000000000000..4b72e41e30093b8321e9f5c589dc225ef2b9495a --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/high-availability/4-logical-backup.md @@ -0,0 +1,40 @@ +--- +title: Logical Backup +summary: Logical Backup +author: Guo Huan +date: 2022-05-09 +--- + +# Logical Backup + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +Data in user tables in the database is backed up to a specified storage medium in a general format. + +## Benefits + +Through logical backup, you can achieve the following purposes: + +- Back up user data to a reliable storage medium to secure data. +- Support cross-version recovery and heterogeneous recovery using a general data format. +- Archive cold data. + +## Description + +MogDB provides the logical backup capability to back up data in user tables to local disk files in text or CSV format and restore the data in homogeneous or heterogeneous databases. + +## Enhancements + +None + +## Constraints + +For details about the restrictions on logical backup, see “Server Tools > [gs_dump](5-gs_dump)” in the *Reference Guide*. + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/high-availability/5-physical-backup.md b/product/en/docs-mogdb/v2.1/characteristic-description/high-availability/5-physical-backup.md new file mode 100644 index 0000000000000000000000000000000000000000..fd53dff466a3b4e98acb772b246eb89c9ad31ed5 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/high-availability/5-physical-backup.md @@ -0,0 +1,52 @@ +--- +title: Physical Backup +summary: Physical Backup +author: Guo Huan +date: 2022-05-09 +--- + +# Physical Backup + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +Data in the entire database is backed up to a specified storage medium in an internal format. + +## Benefits + +Through physical backup, you can achieve the following purposes: + +- Back up data of the entire database to a reliable storage medium, improving system reliability. +- Improve backup and restoration performance using an internal data format. +- Archive cold data. + +The typical physical backup policy and application scenario are as follows: + +- On Monday, perform a full backup of the database. +- On Tuesday, perform an incremental backup based on the full backup on Monday. +- On Wednesday, perform an incremental backup based on the incremental backup on Tuesday. +- … +- On Sunday, perform an incremental backup based on the incremental backup on Saturday. + +The preceding backup operations are executed every week. + +## Description + +MogDB 1.1.0 provides the physical backup capability to back up data of the entire database to local disk files, OBS objects, NBU objects, or EISOO objects in the internal database format, and restore data of the entire database in a homogeneous database. In addition to the preceding functions, it also provides advanced functions such as compression, flow control, and resumable backup. + +Physical backup is classified into full backup and incremental backup. The difference is as follows: Full backup includes the full data of the database at the backup time point. The time required for full backup is long (in direct proportion to the total data volume of the database), and a complete database can be restored. An incremental backup involves only incremental data modified after a specified time point. It takes a short period of time (in direct proportion to the incremental data volume and irrelevant to the total data volume). However, a complete database can be restored only after the incremental backup and full backup are performed. + +## Enhancements + +Supports full backup and incremental backup simultaneously. + +## Constraints + +For details about constraints on physical backup, see “[Physical Backup and Restoration](1-2-br)” in the *Administrator Guide*. + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/high-availability/6-automatic-job-retry-upon-failure.md b/product/en/docs-mogdb/v2.1/characteristic-description/high-availability/6-automatic-job-retry-upon-failure.md new file mode 100644 index 0000000000000000000000000000000000000000..8bc64b1cc6ec22e70898ac958b945126b4d38500 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/high-availability/6-automatic-job-retry-upon-failure.md @@ -0,0 +1,87 @@ +--- +title: Automatic Job Retry upon Failure +summary: Automatic Job Retry upon Failure +author: Guo Huan +date: 2022-05-09 +--- + +# Automatic Job Retry upon Failure + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +If an error occurs in batch processing jobs due to network exceptions or deadlocks, failed jobs are automatically retried. + +## Benefits + +In common fault scenarios, such as network exception and deadlock, queries retry automatically in case of failure to improve database usability. + +## Description + +MogDB provides the job retry mechanism: gsql Retry. + +- The gsql retry mechanism uses a unique error code (SQL STATE) to identify an error that requires a retry. The function of the client tool gsql is enhanced. The error code configuration file **retry_errcodes.conf** is used to configure the list of errors that require a retry. The file is stored in the installation directory at the same level as gsql. **gsql** provides the `set RETRY [number]` command to enable or disable the retry function. The number of retry times ranges from 5 to 10, and the default value is **5**. When this function is enabled, **gsql** reads the preceding configuration file. The error retry controller records the error code list through the container. If an error occurs in the configuration file after the function is enabled, the controller sends the cached query statement to the server for retry until the query is successful or an error is reported when the number of retry times exceeds the maximum. + +## Enhancements + +None + +## Constraints + +- Functionality constraints: + + - Retrying increases execution success rate but does not guarantee success. + +- Error type constraints: + + Only the error types in Table 1 are supported. + + **Table 1** Supported error types + + | Error Type | Error Code | Remarks | + | :-------------------------------- | :--------- | :----------------------------------------------------------- | + | CONNECTION_RESET_BY_PEER | YY001 | TCP communication error. Print information: “Connection reset by peer” | + | STREAM_CONNECTION_RESET_BY_PEER | YY002 | TCP communication error. Print information: “Stream connection reset by peer” (communication between DNs) | + | LOCK_WAIT_TIMEOUT | YY003 | Lock wait timeout. Print information: “Lock wait timeout” | + | CONNECTION_TIMED_OUT | YY004 | TCP communication error. Print information: “Connection timed out” | + | SET_QUERY_ERROR | YY005 | Failed to deliver the **SET** command. Print information: “Set query error” | + | OUT_OF_LOGICAL_MEMORY | YY006 | Failed to apply for memory. Print information: “Out of logical memory” | + | SCTP_MEMORY_ALLOC | YY007 | SCTP communication error. Print information: “Memory allocate error” | + | SCTP_NO_DATA_IN_BUFFER | YY008 | SCTP communication error. Print information: “SCTP no data in buffer” | + | SCTP_RELEASE_MEMORY_CLOSE | YY009 | SCTP communication error. Print information: “Release memory close” | + | SCTP_TCP_DISCONNECT | YY010 | SCTP and TCP communication error. Print information: “SCTP, TCP disconnect” | + | SCTP_DISCONNECT | YY011 | SCTP communication error. Print information: “SCTP disconnect” | + | SCTP_REMOTE_CLOSE | YY012 | SCTP communication error. Print information: “Stream closed by remote” | + | SCTP_WAIT_POLL_UNKNOW | YY013 | Waiting for an unknown poll. Print information: “SCTP wait poll unknow” | + | SNAPSHOT_INVALID | YY014 | Invalid snapshot. Print information: “Snapshot invalid” | + | ERRCODE_CONNECTION_RECEIVE_WRONG | YY015 | Failed to receive a connection. Print information: “Connection receive wrong” | + | OUT_OF_MEMORY | 53200 | Out of memory. Print information: “Out of memory” | + | CONNECTION_EXCEPTION | 08000 | Failed to communicate with DNs due to connection errors. Print information: “Connection exception” | + | ADMIN_SHUTDOWN | 57P01 | System shutdown by the administrator. Print information: “Admin shutdown” | + | STREAM_REMOTE_CLOSE_SOCKET | XX003 | Remote socket disabled. Print information: “Stream remote close socket” | + | ERRCODE_STREAM_DUPLICATE_QUERY_ID | XX009 | Duplicate query. Print information: “Duplicate query id” | + | ERRCODE_STREAM_CONCURRENT_UPDATE | YY016 | Concurrent stream query and update. Print information: “Stream concurrent update” | + +- Statement type constraints: + + Support single-statement stored procedures, functions, and anonymous blocks. Statements in transaction blocks are not supported. + +- Statement constraints of a stored procedure: + + - If an error occurs during the execution of a stored procedure containing EXCEPTION (including statement block execution and statement execution in EXCEPTION), the stored procedure can be retried. If the error is captured by EXCEPTION, the stored procedure cannot be retried. + - Advanced packages that use global variables are not supported. + - DBE_TASK is not supported. + - PKG_UTIL file operation is not supported. + +- Data import constraints: + + - The **COPY FROM STDIN** statement is not supported. + - The **gsql \copy from** metacommand is not supported. + - Data cannot be imported using **JDBC CopyManager copyIn**. + +## Dependencies + +Valid only if the **gsql** tool works normally and the error list is correctly configured. \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/high-availability/7-ultimate-rto.md b/product/en/docs-mogdb/v2.1/characteristic-description/high-availability/7-ultimate-rto.md new file mode 100644 index 0000000000000000000000000000000000000000..c3eaf1339a8fe6483df073d25302715c065633fc --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/high-availability/7-ultimate-rto.md @@ -0,0 +1,39 @@ +--- +title: Ultimate RTO +summary: Ultimate RTO +author: Guo Huan +date: 2022-05-09 +--- + +# Ultimate RTO + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +- The database host can be quickly restored after being restarted. +- Logs can be synchronized between the primary and standby nodes to accelerate playback on the standby node. + +## Benefits + +When the service load is heavy, the playback speed of the standby node cannot catch up with that of the primary node. After the system runs for a long time, logs are accumulated on the standby node. If a host is faulty, data restoration takes a long time and the database is unavailable, which severely affects system availability. + +The ultimate recovery time object (RTO) is enabled to reduce the data recovery time after a host fault occurs and improve availability. + +## Description + +After the ultimate RTO function is enabled, multi-level pipelines are established for Xlog log playback to improve the concurrency and log playback speed. + +## Enhancements + +None + +## Constraints + +The ultimate RTO focuses only on whether the RTO of the standby node meets the requirements. The ultimate RTO has the flow control effect. Therefore, you do not need to enable the flow control function. This feature does not support the read operation on the standby node. If you query the standby node, a core dump may occur on the standby node. This feature does not apply to the primary/standby/secondary scenario. In primary/standby/secondary deployment mode (that is, **replication_type** is set to **0**), the database cannot be started. + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/high-availability/8-cascaded-standby-server.md b/product/en/docs-mogdb/v2.1/characteristic-description/high-availability/8-cascaded-standby-server.md new file mode 100644 index 0000000000000000000000000000000000000000..4cf867837d3c0ecfdc48804d7749212f9c7e19d0 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/high-availability/8-cascaded-standby-server.md @@ -0,0 +1,45 @@ +--- +title: Cascaded Standby Server +summary: Cascaded Standby Server +author: Guo Huan +date: 2022-05-09 +--- + +# Cascaded Standby Server + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +A cascaded standby server can be connected to a standby server based on the one-primary-multiple-standby architecture. + +## Benefits + +The one-primary-multiple-standby architecture cannot support a flexible structure in feature service scenarios. The multi-equipment room deployment cannot meet requirements of the complete structure in the HA switchover scenario (three instances in the primary and standby equipment rooms and two or three instances in the secondary equipment room). If the number of standby servers increases, the primary server may be overloaded. Queries that have low real-time requirements can be implemented on cascaded standby servers. Therefore, the cascading backup capability is required. + +## Description + +The primary server replicates logs to the standby server in synchronous or asynchronous mode. The standby server replicates logs to the cascaded standby server only in asynchronous mode. + +In the current one-primary-multiple-standby architecture, the primary server uses the WAL sender process (walsender) to replicate logs to the standby server. The standby server uses the WAL receiver process (walreceiver) to receive and then flushes logs to local disks. The standby server reads redo logs to complete data replication between the primary and standby servers. There is a one-to-one mapping between walsender and walreceiver on the primary and standby servers. Logs are sent between the standby and cascaded standby servers in asynchronous mode using walsender and walreceiver, reducing the streaming replication pressure on the primary server. + +## Enhancements + +None + +## Constraints + +- A cascaded standby server can only replicate data from a standby server and cannot directly replicate data from the primary server. +- A cascaded standby server does not support data build from a standby server. Currently, data can be built only from the primary server. If the standby server is fully built, the cascaded standby server needs to be fully built. +- The cascaded standby node is in asynchronous replication mode. +- The cascaded standby server cannot be promoted. +- The cascaded standby server cannot be notified. +- Currently, the overall architecture of the primary-standby-cascaded standby cluster cannot be queried. You need to find the standby server through the primary server and then find the cascaded standby server based on the standby server. +- A cascaded standby server cannot own another cascaded standby server. +- When the ultimate RTO is enabled, no cascaded standby server is supported. + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/high-availability/9-delayed-replay.md b/product/en/docs-mogdb/v2.1/characteristic-description/high-availability/9-delayed-replay.md new file mode 100644 index 0000000000000000000000000000000000000000..d9d84e8ddd3b13f91e468f980092884ee9a2919b --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/high-availability/9-delayed-replay.md @@ -0,0 +1,46 @@ +--- +title: Delayed Replay +summary: Delayed Replay +author: Guo Huan +date: 2022-05-09 +--- + +# Delayed Replay + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +The time for a standby node to replay can be delayed. + +## Benefits + +By default, the standby server restores the Xlog records from the primary server as soon as possible. This function allows you to delay the time for a standby node to replay Xlog records. In this case, you can query a copy that records data before a period of time, which helps correct errors such as misoperations. + +## Description + +The GUC parameter **recovery_min_apply_delay** can be used to set the delay time so that a standby server can replay Xlog records from the primary server after a delay time. + +Value range: an integer ranging from 0 to INT_MAX. The unit is ms. + +Default value: **0** (no delay) + +## Enhancements + +None + +## Constraints + +- The **recovery_min_apply_delay** parameter is invalid on the primary node. It must be set on the standby node to be delayed. +- The delay time is calculated based on the timestamp of transaction commit on the primary server and the current time on the standby server. Therefore, ensure that the clocks of the primary and standby servers are the same. +- Operations without transactions are not delayed. +- After the primary/standby switchover, if the original primary server needs to be delayed, you need to manually set this parameter. +- When **synchronous_commit** is set to **remote_apply**, synchronous replication is affected by the delay. Each commit message is returned only after the replay on the standby server is complete. +- Using this feature also delays **hot_standby_feedback**, which may cause the primary server to bloat, so be careful when using both. +- If a DDL operation (such as DROP or TRUNCATE) that holds an AccessExclusive lock is performed on the primary server, the query operation on the operation object on the standby server will be returned only after the lock is released during the delayed replay of the record on the standby server. + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/1-cbo-optimizer.md b/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/1-cbo-optimizer.md new file mode 100644 index 0000000000000000000000000000000000000000..dcc16066c53143a2d471c4796bc2ecab454edf67 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/1-cbo-optimizer.md @@ -0,0 +1,36 @@ +--- +title: CBO Optimizer +summary: CBO Optimizer +author: Guo Huan +date: 2022-05-09 +--- + +# CBO Optimizer + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +The MogDB optimizer is cost-based optimization (CBO). + +## Benefits + +The MogDB CBO optimizer can select the most efficient execution plan among multiple plans based on the cost to meet customer service requirements to the maximum extent. + +## Description + +By using CBO, the database calculates the number of tuples and the execution cost for each step under each execution plan based on the number of table tuples, column width, null record ratio, and characteristic values, such as distinct, MCV, and HB values, and certain cost calculation methods. The database then selects the execution plan that takes the lowest cost for the overall execution or for the return of the first tuple. + +## Enhancements + +None + +## Constraints + +None + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/10-xlog-no-lock-flush.md b/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/10-xlog-no-lock-flush.md new file mode 100644 index 0000000000000000000000000000000000000000..d11e3ccc23fdd60f2f54eefe7dda47ce5f6a25e8 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/10-xlog-no-lock-flush.md @@ -0,0 +1,36 @@ +--- +title: Xlog no Lock Flush +summary: Xlog no Lock Flush +author: Guo Huan +date: 2022-05-09 +--- + +# Xlog no Lock Flush + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +Canceled the WalInsertLock contention and dedicated WalWriter disk write thread. + +## Benefits + +The system performance is further improved on the basis that the original Xlog functions remain unchanged. + +## Description + +This feature optimizes the WalInsertLock mechanism by using log sequence numbers (LSNs) and log record counts (LRCs) to record the copy progress of each backend. The backend can directly copy logs to the WalBuffer without contending for the WalInsertLock. In addition, a dedicated WALWriter thread is used to write logs, and the backend thread does not need to ensure the Xlog flushing. + +## Enhancements + +None. + +## Constraints + +None. + +## Dependencies + +None. \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/11-parallel-page-based-redo-for-ustore.md b/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/11-parallel-page-based-redo-for-ustore.md new file mode 100644 index 0000000000000000000000000000000000000000..b1a9293766b2541b1c2ed49690218662246758bf --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/11-parallel-page-based-redo-for-ustore.md @@ -0,0 +1,36 @@ +--- +title: Parallel Page-based Redo For Ustore +summary: Parallel Page-based Redo For Ustore +author: Guo Huan +date: 2022-05-09 +--- + +# Parallel Page-based Redo For Ustore + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +Optimized Ustore inplace update WAL write and improved the degree of parallelism for Ustore DML operation replay. + +## Benefits + +The WAL space used by the update operation is reduced, and the degree of parallelism for Ustore DML operation replay is improved. + +## Description + +Prefixes and suffixes are used to reduce the write times of WAL update. Replay threads are classified to solve the problem that most Ustore DML WALs are replayed on multiple pages. In addition, Ustore data pages are replayed based on **blkno**. + +## Enhancements + +None. + +## Constraints + +None. + +## Dependencies + +This feature depends on the Ustore engine. \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/2-llvm.md b/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/2-llvm.md new file mode 100644 index 0000000000000000000000000000000000000000..ff02dd18eb13a4141655848ff871d3b7f03bfef0 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/2-llvm.md @@ -0,0 +1,36 @@ +--- +title: LLVM +summary: LLVM +author: Guo Huan +date: 2022-05-09 +--- + +# LLVM + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +MogDB provides the Low Level Virtual Machine (LLVM) technology to query dynamic compilation execution. + +## Benefits + +The requery performance is greatly improved by dynamically building and executing queries. + +## Description + +Based on the query execution plan tree, with the database functions provided by the LLVM, MogDB moves the process of determining the actual execution path from the executor phase to the execution initialization phase. In this way, problems such as function calling, logic condition branch determination, and a large amount of data reading that are related to the original query execution are avoided, to improve the query performance. + +## Enhancements + +None + +## Constraints + +None + +## Dependencies + +It depends on the LLVM open-source component. Currently, the open-source version 10.0.0 is used. \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/3-vectorized-engine.md b/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/3-vectorized-engine.md new file mode 100644 index 0000000000000000000000000000000000000000..d236b4a9030bba4cbe850daea2e5d46803181f0b --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/3-vectorized-engine.md @@ -0,0 +1,43 @@ +--- +title: Vectorized Engine +summary: Vectorized Engine +author: Guo Huan +date: 2022-05-09 +--- + +# Vectorized Engine + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +The vectorized execution engine, provided by MogDB, is usually used in OLAP data warehouse systems because analytical systems are usually data-intensive and access most data in a table in a sequential manner, perform calculation, and finally output a calculation result to an end user. + +## Benefits + +Batch calculation greatly improves the performance of complex query. + +## Description + +The traditional database query execution uses the tuple-based pipeline execution mode. In most time, the CPU is not used to actually process data, but to traverse the query operation tree. As a result, the effective utilization of the CPU is not high. This also results in low instruction cache performance and frequent jumps. Worse still, this approach does not take advantage of the new capabilities of the new hardware to speed up the execution of queries. In the execution engine, another solution is to change a tuple to a column at a time. This is also the basis of our vectorized execution engine. + +The vectorized engine is bound to the column-store technology, because data of each column is stored together, and it may be considered that the data is stored in an array manner. Based on such a feature, when a same operation needs to be performed on the column data, calculation of each value of the data block may be efficiently completed by using a cycle. + +The advantages of the vectorized execution engine are as follows: + +- This reduces inter-node scheduling and improves CPU usage. +- Because the same type of data is put together, it is easier to leverage the new optimization features of hardware and compilation. + +## Enhancements + +None + +## Constraints + +None + +## Dependencies + +It depends on column store. \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/4-hybrid-row-column-store.md b/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/4-hybrid-row-column-store.md new file mode 100644 index 0000000000000000000000000000000000000000..63604d06a6ee7db5ec573e0bc33244f7c471dd26 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/4-hybrid-row-column-store.md @@ -0,0 +1,72 @@ +--- +title: Hybrid Row-Column Store +summary: Hybrid Row-Column Store +author: Guo Huan +date: 2022-05-09 +--- + +# Hybrid Row-Column Store + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +MogDB supports both row-store and column-store models. Choose a row-store or column-store table as needed. + +Column-store is recommended if a table contains many columns (called a wide table) but its query involves only a few columns. Figure 1 shows the column-store model. Row store is recommended if a table contains only a few columns and a query involves most of the fields. + +**Figure 1** Column-store + + ![img](https://cdn-mogdb.enmotech.com/docs-media/mogdb/characteristic-description/hybrid-row-column-store-2.png) + +## Benefits + +In a wide table containing a huge amount of data, a query usually only includes certain columns. In this case, the query performance of the row-store engine is poor. For example, a single table containing the data of a meteorological agency has 200 to 800 columns. Among these columns, only 10 are frequently accessed. In this case, a vectorized execution and column-store engine can significantly improve performance by saving storage space. + +## Description + +Tables are categorized into row-store and column-store tables. Each storage model applies to specific scenarios. Select an appropriate model when creating a table. + +- Row-store table + + Row-store tables are created by default. Data is stored by row. Row-store supports adding, deleting, modifying, and querying data of a complete row. Therefore, this storage model applies to scenarios where data needs to be updated frequently. + +- Column-store table + + Data is stored by column. The I/O of data query in a single column is small, and column-store tables occupy less storage space than row-store tables. This storage model applies to scenarios where data is inserted in batches, less updated, and queried for statistical analysis. The performance of single point query and single record insertion in a column-store table is poor. + +- Selecting a storage model + + - Update frequency + + If data is frequently updated, use a row-store table. + + - Data insertion frequency + + If a small amount of data is frequently inserted each time, use a row-store table. If a large amount of data is inserted at a time, use a column-store table. + + - Number of columns + + If a table is to contain many columns, use a column-store table. + + - Number of columns to be queried + + If only a small number of columns (less than 50% of the total) is queried each time, use a column-store table. + + - Compression ratio + + The compression ratio of a column-store table is higher than that of a row-store table. High compression ratio consumes more CPU resources. + +## Enhancements + +None + +## Constraints + +None + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/5-adaptive-compression.md b/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/5-adaptive-compression.md new file mode 100644 index 0000000000000000000000000000000000000000..6d73315abebbe25e3a4a755e02b6f8ad2dc4f23c --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/5-adaptive-compression.md @@ -0,0 +1,42 @@ +--- +title: Adaptive Compression +summary: Adaptive Compression +author: Guo Huan +date: 2022-05-09 +--- + +# Adaptive Compression + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +Data compression is the major technology used in current databases. Various compression algorithms are used for different data types. If pieces of data of the same type have different characteristics, their compression algorithms and results will also be different. Adaptive compression chooses the suitable compression algorithm for data based on the data type and characteristics, achieving high performance in compression ratio, import, and query. + +## Benefits + +Importing and frequently querying a huge amount of data are the main application scenarios. When you import data, adaptive compression greatly reduces the data volume, increases I/O operation efficiency several times, and clusters data before storage, achieving fast data import. In this way, only a small number of I/O operations is required and data is quickly decompressed in a query. Data can be quickly retrieved and the query result is quickly returned. + +## Description + +Currently, the database has implemented various compression algorithms on column store, including RLE, DELTA, BYTEPACK/BITPACK, LZ4, ZLIB, and LOCAL DICTIONARY. The following table lists data types and the compression algorithms suitable for them. + +| - | RLE | DELTA | BITPACK/BYTEPACK | LZ4 | ZLIB | LOCAL DICTIONARY | +| :----------------------------------------------------------- | :--- | :---- | :--------------- | :--- | :--- | :--------------- | +| Smallint/Int/Bigint/OidDecimal/Real/DoubleMoney/Time/Date/Timestamp | √ | √ | √ | √ | √ | - | +| Tinterval/Interval/Time with time zone/ | - | - | - | - | √ | - | +| Numeric/Char/Varchar/Text/Nvarchar2and other supported data types | √ | √ | √ | √ | √ | √ | + +## Enhancements + +The compression level of compression algorithms can be adjusted. + +## Constraints + +None + +## Dependencies + +It depends on LZ4 or ZLIB. \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/6-sql-by-pass.md b/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/6-sql-by-pass.md new file mode 100644 index 0000000000000000000000000000000000000000..4f7bb1e383397ee16e4d8622965284428b3048c9 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/6-sql-by-pass.md @@ -0,0 +1,36 @@ +--- +title: SQL by pass +summary: SQL by pass +author: Guo Huan +date: 2022-05-09 +--- + +# SQL by pass + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +Query performance is improved by customizing an execution scheme for typical queries in the TP scenario. + +## Benefits + +The TP query performance is improved. + +## Description + +In a typical OLTP scenario, simple queries account for a large proportion. This type of queries involves only single tables and simple expressions. To accelerate such query, the SQL bypass framework is proposed. After simple mode judgment is performed on such query at the parse layer, the query enters a special execution path and skips the classic execution framework, including operator initialization and execution, expression, and projection. Instead, it directly rewrites a set of simple execution paths and directly invokes storage interfaces, greatly accelerating the execution of simple queries. + +## Enhancements + +None + +## Constraints + +None + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/7-kunpeng-numa-architecture-optimization.md b/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/7-kunpeng-numa-architecture-optimization.md new file mode 100644 index 0000000000000000000000000000000000000000..aaf31d2afcef4772e11966a10a3961930d643582 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/7-kunpeng-numa-architecture-optimization.md @@ -0,0 +1,40 @@ +--- +title: Kunpeng NUMA Architecture Optimization +summary: Kunpeng NUMA Architecture Optimization +author: Guo Huan +date: 2022-05-09 +--- + +# Kunpeng NUMA Architecture Optimization + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +Kunpeng NUMA architecture optimization mainly focuses on Kunpeng processor architecture features and ARMv8 instruction set, and optimizes the system from multiple layers, including OS, software architecture, lock concurrency, logs, atomic operations, and cache access. This greatly improves the MogDB performance on the Kunpeng platform. + +## Benefits + +Transactions per minute (TPM) is a key performance indicator of the database competitiveness. Under the same hardware costs, a higher database performance means the database can process more services, thereby reducing the usage cost of customers. + +## Description + +- MogDB optimizes the Kunpeng NUMA architecture based on the architecture characteristics. This reduces cross-core memory access latency and maximizes multi-core Kunpeng computing capabilities. The key technologies include redo log batch insertion, NUMA distribution of hotspot data, and CLog partitions, greatly improving the TP system performance. +- Based on the ARMv8.1 architecture used by the Kunpeng chip, MogDB uses the LSE instruction set to implement efficient atomic operations, effectively improving the CPU usage, multi-thread synchronization performance, and Xlog write performance. +- Based on the wider L3 cache line provided by the Kunpeng chip, MogDB optimizes hotspot data access, effectively improving the cache access hit ratio, reducing the cache consistency maintenance overhead, and greatly improving the overall data access performance of the system. +- Kunpeng 920, 2P server (64 cores x 2, memory: 768 GB), 10 GE network, I/O: 4 NVMe PCIe SSDs, TPC-C: 1000 warehouses, performance: 1,500,000 tpmC. + +## Enhancements + +- Batch redo log insertion and CLog partition are supported, improving the database performance on the Kunpeng platform. +- Efficient atomic operations using the LSE instruction set are supported, improving multi-thread synchronization performance. + +## Constraints + +None + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/8-high-concurrency-of-thread-pools.md b/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/8-high-concurrency-of-thread-pools.md new file mode 100644 index 0000000000000000000000000000000000000000..1aac56b93c0bc8fd8e8486b4fd79709909c85d91 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/8-high-concurrency-of-thread-pools.md @@ -0,0 +1,38 @@ +--- +title: High Concurrency of Thread Pools +summary: High Concurrency of Thread Pools +author: Guo Huan +date: 2022-05-09 +--- + +# High Concurrency of Thread Pools + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +The thread pooling technology is used to support stable running of databases at high concurrency. + +## Benefits + +The overall system throughput is stable in case of a large number of concurrent requests. + +## Description + +The overall design idea of the thread pool technology is to pool thread resources and reuse them among different connections. After the system is started, a fixed number of working threads are started based on the current number of cores or user configuration. A working thread serves one or more connection sessions. In this way, the session and thread are decoupled. The number of worker threads is fixed. Therefore, frequent thread switchover does not occur in case of high concurrency. The database layer schedules and manages sessions. + +## Enhancements + +This feature is available since MogDB 1.1.0. + +In MogDB 1.1.0, thread pools can be dynamically scaled in or out. + +## Constraints + +None + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/9-smp-for-parallel-execution.md b/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/9-smp-for-parallel-execution.md new file mode 100644 index 0000000000000000000000000000000000000000..8b366b07e2d6d3646a38afe80b9db67d55d236c5 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/high-performance/9-smp-for-parallel-execution.md @@ -0,0 +1,44 @@ +--- +title: SMP for Parallel Execution +summary: SMP for Parallel Execution +author: Guo Huan +date: 2022-05-09 +--- + +# SMP for Parallel Execution + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +The Symmetric Multi-Processing (SMP) technology of MogDB uses the multi-core CPU architecture of a computer to implement multi-thread parallel computing, fully using CPU resources to improve query performance. + +## Benefits + +Fully utilizes the system multi-core capability to improve requery performance. + +## Description + +In complex query scenarios, a single query takes long time and the system concurrency is low. Therefore, the SMP technology is used to implement operator-level parallel execution, which effectively reduces the query time and improves the query performance and resource utilization. The overall implementation of the SMP technology is as follows: For query operators that can be executed in parallel, data is sliced, multiple working threads are started for computation, and then the results are summarized and returned to the frontend. The data interaction operator **Stream** is added to the SMP architecture to implement data interaction between multiple working threads, ensuring the correctness and integrity of the query. + +## Enhancements + +None. + +## Constraints + +- Index scanning cannot be executed in parallel. +- MergeJoin cannot be executed in parallel. +- WindowAgg order by cannot be executed in parallel. +- The cursor cannot be executed in parallel. +- Queries in stored procedures and functions cannot be executed in parallel. +- Subplans and initplans cannot be queried in parallel, and operators that contain subqueries cannot be executed in parallel, either. +- Query statements that contain the median operation cannot be executed in parallel. +- Queries with global temporary tables cannot be executed in parallel. +- Updating materialized views cannot be executed in parallel. + +## Dependencies + +None. \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/maintainability/1-gray-upgrade.md b/product/en/docs-mogdb/v2.1/characteristic-description/maintainability/1-gray-upgrade.md new file mode 100644 index 0000000000000000000000000000000000000000..f3c6523486947905d961f4c0ea9faeb1595cb256 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/maintainability/1-gray-upgrade.md @@ -0,0 +1,36 @@ +--- +title: Gray Upgrade +summary: Gray Upgrade +author: Guo Huan +date: 2022-05-09 +--- + +# Gray Upgrade + +## Availability + +This feature is available since MogDB 2.0.0. + +## Introduction + +Gray upgrade supports full-service operations. All nodes can be upgraded at a time. + +## Benefits + +Gray upgrade provides an online upgrade mode to ensure that all nodes are upgraded without interrupting services. + +## Description + +Gray upgrade is an online upgrade mode that upgrades all nodes. If only the binary files of the database need to be replaced during the gray upgrade, to minimize the impact on services, the two sets of binary files exist on the same node at the same time, and the soft connection switchover mode is used to switch the process version (one intermittent disconnection within 10 seconds). + +## Enhancements + +None + +## Constraints + +None + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/maintainability/2-workload-diagnosis-report.md b/product/en/docs-mogdb/v2.1/characteristic-description/maintainability/2-workload-diagnosis-report.md new file mode 100644 index 0000000000000000000000000000000000000000..5fd51d1875ad80c80f13695418bf0b31d02d2d72 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/maintainability/2-workload-diagnosis-report.md @@ -0,0 +1,160 @@ +--- +title: WDR +summary: WDR +author: Guo Huan +date: 2022-05-09 +--- + +# WDR + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +The workload diagnosis report \(WDR\) provides database performance diagnosis reports based on the baseline performance and incremental data that reflects performance changes. + +## Benefits + +- The WDR is the main method for diagnosing long-term performance problems. Based on the performance baseline of a snapshot, performance analysis is performed from multiple dimensions, helping DBAs understand the system load, performance of each component, and performance bottlenecks. +- Snapshots are also an important data source for self-diagnosis and self-optimization suggestions on subsequent performance problems. + +## Description + +The WDR generates a performance report between two different time points based on the system performance snapshot data at these time points. The report is used to diagnose database kernel performance faults. + +You can use generate\_wdr\_report\(...\) to generate a performance report based on two performance snapshots. + +The WDR depends on the following two components: + +- Snapshot: The performance snapshot can be configured to collect a certain amount of performance data from the kernel at a specified interval and store the data in the user tablespace. Any snapshot can be used as a performance baseline for comparison with other snapshots. +- WDR Reporter: This tool analyzes the overall system performance based on two snapshots, calculates the changes of more specific performance indicators between the two time points, and generates summarized and detailed performance data. For details, see Table 1 and Table 2. + +**Table 1** Summarized diagnosis report + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Diagnosis Type

+

Description

+

Database Stat

+

Evaluates the load and I/O status of the current database. Load and I/O are the most important indicators of the TP system.

+

The statistics include the number of sessions connected to the database, number of committed and rolled back transactions, number of read disk blocks, number of disk blocks found in the cache, number of rows returned, captured, inserted, updated, and deleted through database query, number of conflicts and deadlocks, usage of temporary files, and I/O read/write time.

+

Load Profile

+

Evaluates the current system load from the time, I/O, transaction, and SQL dimensions.

+

The statistics include the job running elapse time, CPU time, daily transaction quality, logical and physical read volume, read and write I/O times and size, login and logout times, SQL, transaction execution volume, and SQL P80 and P95 response time.

+

Instance Efficiency Percentages

+

Evaluates the cache efficiency of the current system.

+

The statistics include the database cache hit ratio.

+

Events

+

Evaluates the performance of key system kernel resources and key events.

+

The statistics include the number of times that the key events of the database kernel occur and the waiting time.

+

Wait Classes

+

Evaluates the performance of key events in the system.

+

The statistics include the release of the data kernel in the main types of wait events, such as STATUS, LWLOCK_EVENT, LOCK_EVENT, and IO_EVENT.

+

CPU

+

Includes time release of the CPU in user mode, kernel mode, I/O wait mode, or idle mode.

+

IO Profile

+

Includes the number of database I/O times, database I/O data volume, number of redo I/O times, and redo I/O volume.

+

Memory Statistics

+

Includes maximum process memory, used process memory, maximum shared memory, and used shared memory.

+
+ +**Table 2** Detailed diagnosis report + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Diagnosis Type

+

Description

+

Time Model

+

Evaluates the performance of the current system in the time dimension.

+

The statistics include time consumed by the system in each phase, including the kernel time, CPU time, execution time, parsing time, compilation time, query rewriting time, plan generation time, network time, and I/O time.

+

SQL Statistics

+

Diagnoses SQL statement performance problems.

+

The statistics include normalized SQL performance indicators in multiple dimensions: elapsed time, CPU time, rows returned, tuple reads, executions, physical reads, and logical reads. The indicators can be classified into execution time, number of execution times, row activity, and cache I/O.

+

Wait Events

+

Diagnoses performance of key system resources and key time in detail.

+

The statistics include the performance of all key events in a period of time, including the number of events and the time consumed.

+

Cache IO Stats

+

Diagnoses the performance of user tables and indexes.

+

The statistics include read and write operations on all user tables and indexes, and the cache hit ratio.

+

Utility status

+

Diagnoses the background task performance.

+

The statistics include the performance of background tasks such as replication.

+

Object stats

+

Diagnoses the performance of database objects.

+

The statistics include user tables, tables on indexes, index scan activities, as well as insert, update, and delete activities, number of valid rows, and table maintenance status.

+

Configuration settings

+

Determines whether the configuration is changed.

+

It is a snapshot that contains all current configuration parameters.

+

SQL detail

+

Displays information about unique query text.

+
+ +## Enhancements + +None. + +## Constraints + +- The WDR snapshot collects performance data of different databases. If there are a large number of databases or tables in the database instance, it takes a long time to create a WDR snapshot. +- If WDR snapshot is performed when a large number of DDL statements are executed, WDR snapshot may fail. +- When the database is dropped, WDR snapshot may fail. + +## Dependencies + +None. \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/maintainability/3-slow-sql-diagnosis.md b/product/en/docs-mogdb/v2.1/characteristic-description/maintainability/3-slow-sql-diagnosis.md new file mode 100644 index 0000000000000000000000000000000000000000..2f23b862c7d566e51ab906c0bb6f094134b475ea --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/maintainability/3-slow-sql-diagnosis.md @@ -0,0 +1,137 @@ +--- +title: Slow SQL Diagnosis +summary: Slow SQL Diagnosis +author: Guo Huan +date: 2022-05-09 +--- + +# Slow SQL Diagnosis + +## Availability + +This feature is available since MogDB 1.1.0. The following slow SQL views have been discarded before reconstruction: dbe_perf.gs_slow_query_info, dbe_perf.gs_slow_query_history, dbe_perf.global_slow_query_hisotry, and dbe_perf.global_slow_query_info. + +## Introduction + +Slow SQL diagnosis provides necessary information for diagnosing slow SQL statements, helping developers backtrack SQL statements whose execution time exceeds the threshold and diagnose SQL performance bottlenecks. + +## Benefits + +Slow SQL provides detailed information required for slow SQL diagnosis. You can diagnose performance problems of specific slow SQL statements offline without reproducing the problem. The table-based and function-based APIs help users collect statistics on slow SQL indicators and connect to third-party platforms. + +## Description + +Slow SQL diagnosis records information about all jobs whose execution time exceeds the threshold **log_min_duration_statement**. + +Slow SQL provides table-based and function-based query APIs. You can query the execution plan, start time, end time, query statement, row activity, kernel time, CPU time, execution time, parsing time, compilation time, query rewriting time, plan generation time, network time, I/O time, network overhead, and lock overhead. All information is anonymized. + +## Enhancements + +Optimized slow SQL indicators, security (anonymization), execution plans, and query interfaces. + +``` +Copy CodeRun the following command to check the execution information about the SQL statements in the database instance: +gsql> select * from dbe_perf.get_global_full_sql_by_timestamp(start_timestamp, end_timestamp); +For example: +MogDB=# select * from DBE_PERF.get_global_full_sql_by_timestamp('2020-12-01 09:25:22', '2020-12-31 23:54:41'); +-[ RECORD 1 ]--------+--------------------------------------------------------------------------------------------------------------- +--------------------------------------------------------------- +node_name | dn_6001_6002_6003 +db_name | postgres +schema_name | "$user",public +origin_node | 1938253334 +user_name | user_dj +application_name | gsql +client_addr | +client_port | -1 +unique_query_id | 3671179229 +debug_query_id | 72339069014839210 +query | select name, setting from pg_settings where name in (?) +start_time | 2020-12-19 16:19:51.216818+08 +finish_time | 2020-12-19 16:19:51.224513+08 +slow_sql_threshold | 1800000000 +transaction_id | 0 +thread_id | 139884662093568 +session_id | 139884662093568 +n_soft_parse | 0 +n_hard_parse | 1 +query_plan | Datanode Name: dn_6001_6002_6003 + | Function Scan on pg_show_all_settings a (cost=0.00..12.50 rows=5 width=64) + | Filter: (name = '***'::text) +... + +Run the following command to check the execution information about the slow SQL statements in the database instance: +gsql> select * from dbe_perf.get_global_slow_sql_by_timestamp(start_timestamp, end_timestamp); +MogDB=# select * from DBE_PERF.get_global_slow_sql_by_timestamp('2020-12-01 09:25:22', '2020-12-31 23:54:41'); +-[ RECORD 1 ]--------+--------------------------------------------------------------------------------------------------- +node_name | dn_6001_6002_6003 +db_name | postgres +schema_name | "$user",public +origin_node | 1938253334 +user_name | user_dj +application_name | gsql +client_addr | +client_port | -1 +unique_query_id | 2165004317 +debug_query_id | 72339069014839319 +query | select * from DBE_PERF.get_global_slow_sql_by_timestamp(?, ?); +start_time | 2020-12-19 16:23:20.738491+08 +finish_time | 2020-12-19 16:23:20.773714+08 +slow_sql_threshold | 10000 +transaction_id | 0 +thread_id | 139884662093568 +session_id | 139884662093568 +n_soft_parse | 10 +n_hard_parse | 8 +query_plan | Datanode Name: dn_6001_6002_6003 + | Result (cost=1.01..1.02 rows=1 width=0) + | InitPlan 1 (returns $0) + | -> Seq Scan on pgxc_node (cost=0.00..1.01 rows=1 width=64) + | Filter: (nodeis_active AND ((node_type = '***'::"char") OR (node_type = '***'::"char"))) +... + +Check the execution information about the SQL statement on the current node. +gsql> select * from statement_history; +For example: +MogDB=# select * from statement_history; +-[ RECORD 1 ]--------+--------------------------------------------------------------------------------------------------------------- +--------------------------------------------------------------- +db_name | postgres +schema_name | "$user",public +origin_node | 1938253334 +user_name | user_dj +application_name | gsql +client_addr | +client_port | -1 +unique_query_id | 3671179229 +debug_query_id | 72339069014839210 +query | select name, setting from pg_settings where name in (?) +start_time | 2020-12-19 16:19:51.216818+08 +finish_time | 2020-12-19 16:19:51.224513+08 +slow_sql_threshold | 1800000000 +transaction_id | 0 +thread_id | 139884662093568 +session_id | 139884662093568 +n_soft_parse | 0 +n_hard_parse | 1 +query_plan | Datanode Name: dn_6001_6002_6003 + | Function Scan on pg_show_all_settings a (cost=0.00..12.50 rows=5 width=64) + | Filter: (name = '***'::text) +``` + +## Constraints + +- The SQL tracing information is based on the normal execution logic. The tracing information may inaccurate if SQL statements fail to be executed. +- Restarting a node may cause data loss on the node. +- If you exit a session immediately after SQL statements are executed, the session data that is not updated to the system catalog may be lost. +- The number of SQL statements to be collected is specified by a GUC parameter. If the number of SQL statements exceeds the threshold, new SQL statement execution information will not be collected. +- The maximum number of bytes of lock event details collected by a single SQL statement is specified by a GUC parameter. If the number of bytes exceeds the threshold, new lock event details will not be collected. +- The SQL statement information is updated in asynchronous mode. Therefore, after a query statement is executed, the related view function result is slightly delayed. +- Certain indicator information (such as row activities, cache I/O, and time distribution) depends on the dbe_perf.statement view. If the number of records in the view exceeds the preset size (depending on GUC:instr_unique_sql_count), related indicators may not be collected. +- Functions related to the statement_history table and the details column in the view are in binary format. To parse the detailed information, use the **pg_catalog.statement_detail_decode(details, 'plaintext', true)** function. +- The statement_history table can be queried only in the postgres database. The data in other databases is empty. +- The content of the statement_history table is controlled by track_stmt_stat_level. The default value is **'OFF,L0'**. The first part of the parameter indicates the full SQL statement, and the second part indicates the slow SQL statement. Slow SQL statements are recorded in the statement_history table only when the SQL statement execution time exceeds the value of **log_min_duration_statement**. + +## Dependencies + +None. \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/maintainability/4-session-performance-diagnosis.md b/product/en/docs-mogdb/v2.1/characteristic-description/maintainability/4-session-performance-diagnosis.md new file mode 100644 index 0000000000000000000000000000000000000000..8779a816fe73f1b28416e1bb463bd9863b29d6ad --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/maintainability/4-session-performance-diagnosis.md @@ -0,0 +1,102 @@ +--- +title: Session Performance Diagnosis +summary: Session Performance Diagnosis +author: Guo Huan +date: 2022-05-09 +--- + +# Session Performance Diagnosis + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +Session performance diagnosis targets session-level performance faults. + +## Benefits + +- Display the latest events that consume the most resources of user sessions. +- Check the wait events that occupy the most resource-consuming SQL statements. +- Check the wait events that occupy the most resource-consuming sessions. +- Check information about the most resource-consuming users. +- Check the waiting relationship between blocked sessions. + +## Description + +The session performance diagnosis function diagnoses performance of all active sessions in the system. As real-time collection of indicators of all active sessions has a greater impact on user load, the session snapshot technology is used to sample indicators of active sessions, and collect statistics on active sessions from the sampling. The statistics reflect the basic information, status, and resources of active sessions from the dimensions of client information, execution start time, execution end time, SQL text, wait events, and current database objects. The active session information collected based on the probability can help users diagnose which sessions consume more CPU and memory resources, which database objects are hot objects, and which SQL statements consume more key event resources in the system. In this way, users can locate faulty sessions, SQL statements, and database designs. + +Session sampling data is classified into two levels, as shown in Figure 1. + +1. The first level is real-time information stored in the memory. The active session information in the latest several minutes is displayed, which has the highest precision. +2. The second level is the persistent historical information stored in disk files. It displays the historical active session information in a long period of time and is sampled from the memory data. This level is suitable for long-run statistics and analysis. + +**Figure 1** Session performance diagnosis principle + + ![session-performance-diagnosis-principle](https://cdn-mogdb.enmotech.com/docs-media/mogdb/characteristic-description/session-performance-diagnosis-1.png) + +Some application scenarios are as follows: + +1. Check the blocking relationship between sessions. + + ``` + select sessionid, block_sessionid from pg_thread_wait_status; + ``` + +2. Sample information about blocked sessions. + + ``` + select sessionid, block_sessionid from DBE_PERF.local_active_session; + ``` + +3. Display the final blocked session. + + ``` + select sessionid, block_sessionid, final_block_sessionid from DBE_PERF.local_active_session; + ``` + +4. Check the wait event that consumes the most resources. + + ``` + SELECT s.type, s.event, t.count + FROM dbe_perf.wait_events s, ( + SELECT event, COUNT(*) + FROM dbe_perf.local_active_session + WHERE sample_time > now() - 5 / (24 * 60) + GROUP BY event)t WHERE s.event = t.event ORDER BY count DESC; + ``` + +5. Check the events that consume the most session resources in the last five minutes. + + ``` + SELECT sessionid, start_time, event, count + FROM ( + SELECT sessionid, start_time, event, COUNT(*) + FROM dbe_perf.local_active_session + WHERE sample_time > now() - 5 / (24 * 60) + GROUP BY sessionid, start_time, event) as t ORDER BY SUM(t.count) OVER (PARTITION BY t. sessionid, start_time)DESC, t.event; + ``` + +6. Check the events that consume the most resources in the last five minutes. + + ``` + SELECT query_id, event, count + FROM ( + SELECT query_id, event, COUNT(*) + FROM dbe_perf.local_active_session + WHERE sample_time > now() - 5 / (24 * 60) + GROUP BY query_id, event) t ORDER BY SUM(t.count) OVER (PARTITION BY t.query_id ) DESC, t.event DESC; + ``` + +## Enhancements + +None + +## Constraints + +None + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/characteristic-description/maintainability/5-system-kpi-aided-diagnosis.md b/product/en/docs-mogdb/v2.1/characteristic-description/maintainability/5-system-kpi-aided-diagnosis.md new file mode 100644 index 0000000000000000000000000000000000000000..ff39707d5eb1d334e437fd261e9aeaabd580db26 --- /dev/null +++ b/product/en/docs-mogdb/v2.1/characteristic-description/maintainability/5-system-kpi-aided-diagnosis.md @@ -0,0 +1,66 @@ +--- +title: System KPI-aided Diagnosis +summary: System KPI-aided Diagnosis +author: Guo Huan +date: 2022-05-09 +--- + +# System KPI-aided Diagnosis + +## Availability + +This feature is available since MogDB 1.1.0. + +## Introduction + +KPIs are views of key performance indicators for kernel components or the entire system. Based on KPIs, users can learn about the real-time and historical running status of the system. + +## Benefits + +- Summarized system load diagnosis + + Precise alarms for system load exceptions (overload, stall, and SLA exceptions) and precise system load profile + +- Summarized system time model diagnosis + + Instance-level and query-level time model segmentation, diagnosing the root causes of instance and query performance problems + +- Query performance diagnosis + + Database-level query summary, including top SQL, SQL CPU usage, I/O consumption, execution plan, and excessive hard parsing + +- Diagnosis of disk I/O, index, and buffer performance problems + +- Diagnosis of connection and thread pool problems + +- Diagnosis of checkpoint and redo (RTO) performance problems + +- Diagnosis of system I/O, LWLock, and wait performance problems + + Diagnosis of over 60 modules and over 240 key operation performance problems + +- Function-level performance monitoring and diagnosis (by GSTRACE) + + Tracing of over 50 functions at the storage and execution layers + +## Description + +MogDB provides KPIs of 11 categories and 26 sub-categories, covering instances, files, objects, workload, communication, sessions, threads, cache I/O, locks, wait events, and clusters. + +Figure 1 shows the distribution of kernel KPIs. + +**Figure 1** Distribution of kernel KPIs + + ![distribution-of-kernel-kpis](https://cdn-mogdb.enmotech.com/docs-media/mogdb/characteristic-description/system-kpi-aided-diagnosis-2.png) + +## Enhancements + +None + +## Constraints + +None + +## Dependencies + +None \ No newline at end of file diff --git a/product/en/docs-mogdb/v2.1/toc.md b/product/en/docs-mogdb/v2.1/toc.md index 7b3f83a8bd09b1424451bde47232c30333fe40f0..77589c1ea63c18241bfae68048c4c6ba256b2c43 100644 --- a/product/en/docs-mogdb/v2.1/toc.md +++ b/product/en/docs-mogdb/v2.1/toc.md @@ -45,6 +45,78 @@ + [Rust](/quick-start/mogdb-access/use-programming-language-to-access-mogdb/rust.md) + [NodeJS](/quick-start/mogdb-access/use-programming-language-to-access-mogdb/nodejs.md) + [Using Sample Dataset Mogila](/quick-start/mogila.md) ++ Characteristic Description + + High Performance + + [CBO Optimizer](/characteristic-description/high-performance/1-cbo-optimizer.md) + + [LLVM](/characteristic-description/high-performance/2-llvm.md) + + [Vectorized Engine](/characteristic-description/high-performance/3-vectorized-engine.md) + + [Hybrid Row-Column Store](/characteristic-description/high-performance/4-hybrid-row-column-store.md) + + [Adaptive Compression](/characteristic-description/high-performance/5-adaptive-compression.md) + + [SQL by pass](/characteristic-description/high-performance/6-sql-by-pass.md) + + [Kunpeng NUMA Architecture Optimization](/characteristic-description/high-performance/7-kunpeng-numa-architecture-optimization.md) + + [High Concurrency of Thread Pools](/characteristic-description/high-performance/8-high-concurrency-of-thread-pools.md) + + [SMP for Parallel Execution](/characteristic-description/high-performance/9-smp-for-parallel-execution.md) + + [Xlog no Lock Flush](/characteristic-description/high-performance/10-xlog-no-lock-flush.md) + + [Parallel Page-based Redo For Ustore](/characteristic-description/high-performance/11-parallel-page-based-redo-for-ustore.md) + + High Availability (HA) + + [Primary/Standby](/characteristic-description/high-availability/1-primary-standby.md) + + [Logical Replication](/characteristic-description/high-availability/2-logical-replication.md) + + [Online Node Replacement](/characteristic-description/high-availability/3-online-node-replacement.md) + + [Logical Backup](/characteristic-description/high-availability/4-logical-backup.md) + + [Physical Backup](/characteristic-description/high-availability/5-physical-backup.md) + + [Automatic Job Retry upon Failure](/characteristic-description/high-availability/6-automatic-job-retry-upon-failure.md) + + [Ultimate RTO](/characteristic-description/high-availability/7-ultimate-rto.md) + + [Cascaded Standby Server](/characteristic-description/high-availability/8-cascaded-standby-server.md) + + [Delayed Replay](/characteristic-description/high-availability/9-delayed-replay.md) + + [Adding or Deleting a Standby Server](/characteristic-description/high-availability/10-adding-or-deleting-a-standby-server.md) + + Maintainability + + [Gray Upgrade](/characteristic-description/maintainability/1-gray-upgrade.md) + + [Workload Diagnosis Report (WDR)](/characteristic-description/maintainability/2-workload-diagnosis-report.md) + + [Slow SQL Diagnosis](/characteristic-description/maintainability/3-slow-sql-diagnosis.md) + + [Session Performance Diagnosis](/characteristic-description/maintainability/4-session-performance-diagnosis.md) + + [System KPI-aided Diagnosis](/characteristic-description/maintainability/5-system-kpi-aided-diagnosis.md) + + Database Security + + [Access Control Model](/characteristic-description/database-security/1-access-control-model.md) + + [Separation of Control and Access Permissions](/characteristic-description/database-security/2-separation-of-control-and-access-permissions.md) + + [Database Encryption Authentication](/characteristic-description/database-security/3-database-encryption-authentication.md) + + [Data Encryption and Storage](/characteristic-description/database-security/4-data-encryption-and-storage.md) + + [Database Audit](/characteristic-description/database-security/5-database-audit.md) + + [Network Communication Security](/characteristic-description/database-security/6-network-communication-security.md) + + [Resource Label](/characteristic-description/database-security/7-resource-label.md) + + [Unified Audit](/characteristic-description/database-security/8-unified-audit.md) + + [Dynamic Data Anonymization](/characteristic-description/database-security/9-dynamic-data-anonymization.md) + + [Row-Level Access Control](/characteristic-description/database-security/10-row-level-access-control.md) + + [Password Strength Verification](/characteristic-description/database-security/11-password-strength-verification.md) + + [Equality Query in a Fully-encrypted Database](/characteristic-description/database-security/12-equality-query-in-a-fully-encrypted-database.md) + + [Ledger Database Mechanism](/characteristic-description/database-security/13-ledger-database-mechanism.md) + + Enterprise-Level Features + + [Support for Functions and Stored Procedures](/characteristic-description/enterprise-level-features/1-support-for-functions-and-stored-procedures.md) + + [SQL Hints](/characteristic-description/enterprise-level-features/2-sql-hints.md) + + [Full-Text Indexing](/characteristic-description/enterprise-level-features/3-full-text-indexing.md) + + [Copy Interface for Error Tolerance](/characteristic-description/enterprise-level-features/4-copy-interface-for-error-tolerance.md) + + [Partitioning](/characteristic-description/enterprise-level-features/5-partitioning.md) + + [Support for Advanced Analysis Functions](/characteristic-description/enterprise-level-features/6-support-for-advanced-analysis-functions.md) + + [Materialized View](/characteristic-description/enterprise-level-features/7-materialized-view.md) + + [HyperLogLog](/characteristic-description/enterprise-level-features/8-hyperloglog.md) + + [Creating an Index Online](/characteristic-description/enterprise-level-features/9-creating-an-index-online.md) + + [Autonomous Transaction](/characteristic-description/enterprise-level-features/10-autonomous-transaction.md) + + [Global Temporary Table](/characteristic-description/enterprise-level-features/11-global-temporary-table.md) + + [Pseudocolumn ROWNUM](/characteristic-description/enterprise-level-features/12-pseudocolumn-rownum.md) + + [Stored Procedure Debugging](/characteristic-description/enterprise-level-features/13-stored-procedure-debugging.md) + + [JDBC Client Load Balancing and Read/Write Isolation](/characteristic-description/enterprise-level-features/14-jdbc-client-load-balancing-and-readwrite-isolation.md) + + [In-place Update Storage Engine](/characteristic-description/enterprise-level-features/15-in-place-update-storage-engine.md) + + Application Development Interfaces + + [Standard SQL](/characteristic-description/application-development-interfaces/1-standard-sql.md) + + [Standard Development Interfaces](/characteristic-description/application-development-interfaces/2-standard-development-interfaces.md) + + [PostgreSQL API Compatibility](/characteristic-description/application-development-interfaces/3-postgresql-api-compatibility.md) + + [PL/Java](/characteristic-description/application-development-interfaces/4-pl-java.md) + + AI Capabilities + + [Predictor: AI Query Time Forecasting](/characteristic-description/ai-capabilities/1-predictor-ai-query-time-forecasting.md) + + [X-Tuner: Parameter Optimization and Diagnosis](/characteristic-description/ai-capabilities/2-x-tuner-parameter-optimization-and-diagnosis.md) + + [SQLdiag: Slow SQL Discovery](/characteristic-description/ai-capabilities/3-sqldiag-slow-sql-discovery.md) + + [Anomaly-detection: Database Indicator Collection, Forecasting, and Exception Monitoring](/characteristic-description/ai-capabilities/4-anomaly-detection-database-indicator-collection-forecasting-and-exception-monitoring.md) + + [Index-advisor: Index Recommendation](/characteristic-description/ai-capabilities/5-index-advisor-index-recommendation.md) + + [DeepSQL: AI Algorithm in the Database](/characteristic-description/ai-capabilities/6-deepsql-ai-algorithm-in-the-database.md) + Installation Guide + [Container Installation](/installation-guide/docker-installation/docker-installation.md) + Simplified Installation Process diff --git a/product/en/docs-mogdb/v2.1/toc_characteristic_description.md b/product/en/docs-mogdb/v2.1/toc_characteristic_description.md new file mode 100644 index 0000000000000000000000000000000000000000..69e45bc5f8a22ea138c0c0d75007a1bbb751da1c --- /dev/null +++ b/product/en/docs-mogdb/v2.1/toc_characteristic_description.md @@ -0,0 +1,78 @@ + + + +# MogDB Documentation 2.1 + +## 特性描述 + ++ High Performance + + [CBO Optimizer](/characteristic-description/high-performance/1-cbo-optimizer.md) + + [LLVM](/characteristic-description/high-performance/2-llvm.md) + + [Vectorized Engine](/characteristic-description/high-performance/3-vectorized-engine.md) + + [Hybrid Row-Column Store](/characteristic-description/high-performance/4-hybrid-row-column-store.md) + + [Adaptive Compression](/characteristic-description/high-performance/5-adaptive-compression.md) + + [SQL by pass](/characteristic-description/high-performance/6-sql-by-pass.md) + + [Kunpeng NUMA Architecture Optimization](/characteristic-description/high-performance/7-kunpeng-numa-architecture-optimization.md) + + [High Concurrency of Thread Pools](/characteristic-description/high-performance/8-high-concurrency-of-thread-pools.md) + + [SMP for Parallel Execution](/characteristic-description/high-performance/9-smp-for-parallel-execution.md) + + [Xlog no Lock Flush](/characteristic-description/high-performance/10-xlog-no-lock-flush.md) + + [Parallel Page-based Redo For Ustore](/characteristic-description/high-performance/11-parallel-page-based-redo-for-ustore.md) ++ High Availability (HA) + + [Primary/Standby](/characteristic-description/high-availability/1-primary-standby.md) + + [Logical Replication](/characteristic-description/high-availability/2-logical-replication.md) + + [Online Node Replacement](/characteristic-description/high-availability/3-online-node-replacement.md) + + [Logical Backup](/characteristic-description/high-availability/4-logical-backup.md) + + [Physical Backup](/characteristic-description/high-availability/5-physical-backup.md) + + [Automatic Job Retry upon Failure](/characteristic-description/high-availability/6-automatic-job-retry-upon-failure.md) + + [Ultimate RTO](/characteristic-description/high-availability/7-ultimate-rto.md) + + [Cascaded Standby Server](/characteristic-description/high-availability/8-cascaded-standby-server.md) + + [Delayed Replay](/characteristic-description/high-availability/9-delayed-replay.md) + + [Adding or Deleting a Standby Server](/characteristic-description/high-availability/10-adding-or-deleting-a-standby-server.md) ++ Maintainability + + [Gray Upgrade](/characteristic-description/maintainability/1-gray-upgrade.md) + + [Workload Diagnosis Report (WDR)](/characteristic-description/maintainability/2-workload-diagnosis-report.md) + + [Slow SQL Diagnosis](/characteristic-description/maintainability/3-slow-sql-diagnosis.md) + + [Session Performance Diagnosis](/characteristic-description/maintainability/4-session-performance-diagnosis.md) + + [System KPI-aided Diagnosis](/characteristic-description/maintainability/5-system-kpi-aided-diagnosis.md) ++ Database Security + + [Access Control Model](/characteristic-description/database-security/1-access-control-model.md) + + [Separation of Control and Access Permissions](/characteristic-description/database-security/2-separation-of-control-and-access-permissions.md) + + [Database Encryption Authentication](/characteristic-description/database-security/3-database-encryption-authentication.md) + + [Data Encryption and Storage](/characteristic-description/database-security/4-data-encryption-and-storage.md) + + [Database Audit](/characteristic-description/database-security/5-database-audit.md) + + [Network Communication Security](/characteristic-description/database-security/6-network-communication-security.md) + + [Resource Label](/characteristic-description/database-security/7-resource-label.md) + + [Unified Audit](/characteristic-description/database-security/8-unified-audit.md) + + [Dynamic Data Anonymization](/characteristic-description/database-security/9-dynamic-data-anonymization.md) + + [Row-Level Access Control](/characteristic-description/database-security/10-row-level-access-control.md) + + [Password Strength Verification](/characteristic-description/database-security/11-password-strength-verification.md) + + [Equality Query in a Fully-encrypted Database](/characteristic-description/database-security/12-equality-query-in-a-fully-encrypted-database.md) + + [Ledger Database Mechanism](/characteristic-description/database-security/13-ledger-database-mechanism.md) ++ Enterprise-Level Features + + [Support for Functions and Stored Procedures](/characteristic-description/enterprise-level-features/1-support-for-functions-and-stored-procedures.md) + + [SQL Hints](/characteristic-description/enterprise-level-features/2-sql-hints.md) + + [Full-Text Indexing](/characteristic-description/enterprise-level-features/3-full-text-indexing.md) + + [Copy Interface for Error Tolerance](/characteristic-description/enterprise-level-features/4-copy-interface-for-error-tolerance.md) + + [Partitioning](/characteristic-description/enterprise-level-features/5-partitioning.md) + + [Support for Advanced Analysis Functions](/characteristic-description/enterprise-level-features/6-support-for-advanced-analysis-functions.md) + + [Materialized View](/characteristic-description/enterprise-level-features/7-materialized-view.md) + + [HyperLogLog](/characteristic-description/enterprise-level-features/8-hyperloglog.md) + + [Creating an Index Online](/characteristic-description/enterprise-level-features/9-creating-an-index-online.md) + + [Autonomous Transaction](/characteristic-description/enterprise-level-features/10-autonomous-transaction.md) + + [Global Temporary Table](/characteristic-description/enterprise-level-features/11-global-temporary-table.md) + + [Pseudocolumn ROWNUM](/characteristic-description/enterprise-level-features/12-pseudocolumn-rownum.md) + + [Stored Procedure Debugging](/characteristic-description/enterprise-level-features/13-stored-procedure-debugging.md) + + [JDBC Client Load Balancing and Read/Write Isolation](/characteristic-description/enterprise-level-features/14-jdbc-client-load-balancing-and-readwrite-isolation.md) + + [In-place Update Storage Engine](/characteristic-description/enterprise-level-features/15-in-place-update-storage-engine.md) ++ Application Development Interfaces + + [Standard SQL](/characteristic-description/application-development-interfaces/1-standard-sql.md) + + [Standard Development Interfaces](/characteristic-description/application-development-interfaces/2-standard-development-interfaces.md) + + [PostgreSQL API Compatibility](/characteristic-description/application-development-interfaces/3-postgresql-api-compatibility.md) + + [PL/Java](/characteristic-description/application-development-interfaces/4-pl-java.md) ++ AI Capabilities + + [Predictor: AI Query Time Forecasting](/characteristic-description/ai-capabilities/1-predictor-ai-query-time-forecasting.md) + + [X-Tuner: Parameter Optimization and Diagnosis](/characteristic-description/ai-capabilities/2-x-tuner-parameter-optimization-and-diagnosis.md) + + [SQLdiag: Slow SQL Discovery](/characteristic-description/ai-capabilities/3-sqldiag-slow-sql-discovery.md) + + [Anomaly-detection: Database Indicator Collection, Forecasting, and Exception Monitoring](/characteristic-description/ai-capabilities/4-anomaly-detection-database-indicator-collection-forecasting-and-exception-monitoring.md) + + [Index-advisor: Index Recommendation](/characteristic-description/ai-capabilities/5-index-advisor-index-recommendation.md) + + [DeepSQL: AI Algorithm in the Database](/characteristic-description/ai-capabilities/6-deepsql-ai-algorithm-in-the-database.md) \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/ai-capabilities/1-predictor-ai-query-time-forecasting.md b/product/zh/docs-mogdb/v2.1/characteristic-description/ai-capabilities/1-predictor-ai-query-time-forecasting.md new file mode 100644 index 0000000000000000000000000000000000000000..5b5cebff81cb277406af7bd785bb339a99b199fa --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/ai-capabilities/1-predictor-ai-query-time-forecasting.md @@ -0,0 +1,48 @@ +--- +title: Predictor：AI查询时间预测 +summary: Predictor：AI查询时间预测 +author: Guo Huan +date: 2022-05-07 +--- + +# Predictor：AI查询时间预测 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +Predictor是基于机器学习且具有在线学习能力的查询时间预测工具。在数据库内核中，通过不断学习数据库内收集的历史执行信息，实现计划的执行时间预测功能。 + +## 客户价值 + +在SQL语句执行之前，根据AI模型预测SQL执行时间。用户基于推测的SQL时间信息，可以发现可能的异常SQL语句，提前进行调优或者调度，以免影响系统运行。 + +## 特性描述 + +AI查询时间预测，依赖于本地查询计划收集。用户需要根据需求在查询执行时，收集实际查询计划（包括计划结构、算子类型、相关数据源、过滤条件等）、各算子节点实际执行时间、优化器估算代价、实际返回行数、优化器估算行数、并行数等信息。这些记录将保存在数据表中，并进行持久化管理包括定期进行数据失效清理。 + +数据库内核在自动收集历史数据后，由管理员进行编码，然后通过CURL接口向python端发送请求以配置模型、发送数据、触发训练。训练过程中可以通过调用tensorboard接口监控训练过程。模型最终返回各项目预测准确率，并保存最终模型。数据库根据aiEngine端返回结果更新模型信息相关系统表。 + +执行explain (analyze on, predictor ) SELECT …时且模型收敛时触发，数据库根据Plan进行编码判断是否符合当前模型。如果符合模型，先向python端发送加载模型请求（模型训练完成后只需加载一次），模型加载后对计划编码文件进行推理并返回各节点预测结果。 + +## 特性增强 + +无。 + +## 特性约束 + +- 数据库系统处于正常状态，用户通过身份验证成功登陆数据库访问拥有权限的数据。 +- 用户执行的SQL语法正确无报错。 +- 用户执行的SQL不会导致数据库异常等。 +- 历史性能数据窗口内集群并发量稳定，集群规模、表结构、表数量不变，数据量无突变，涉及查询性能的guc参数不变。如果上述条件被破坏，模型失效，需要对所有历史性能数据进行失效并重新进行数据收集和重新训练模型。 +- 加载模型时，原模型的结构、参数和训练信息等保存完整。若原模型信息丢失，模型无法正常加载，需要重新训练。 +- 历史性能数据正常收集和编码，正常解析编码数据不报错。 +- 用户通过预提供的安装脚本或者自行安装Python\==3.6.4，configparser\==3.8.1，Flask\==0.12.2，Keras\==2.2.4，numpy\==1.16.4，scikit-learn\==0.19.1，pandas\==0.25.1， tensorboard\==1.14.0，tensorflow-gpu\==1.14.0或tensorflow==1.14.0。 +- 用户环境中已经安装了OpenSSL，使用脚本或者按照使用说明生成了证书。 +- 目前跨database namespace的模型同步需要管理员用户手动执行，不支持自动同步。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/ai-capabilities/2-x-tuner-parameter-optimization-and-diagnosis.md b/product/zh/docs-mogdb/v2.1/characteristic-description/ai-capabilities/2-x-tuner-parameter-optimization-and-diagnosis.md new file mode 100644 index 0000000000000000000000000000000000000000..fa58addce0b39c0bfa7c6c4ed086565f79307285 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/ai-capabilities/2-x-tuner-parameter-optimization-and-diagnosis.md @@ -0,0 +1,46 @@ +--- +title: X-Tuner：参数调优与诊断 +summary: X-Tuner：参数调优与诊断 +author: Guo Huan +date: 2022-05-07 +--- + +# X-Tuner：参数调优与诊断 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +X-Tuner 是一款数据库集成的参数调优工具，通过结合深度强化学习和全局搜索算法等AI技术，实现在无需人工干预的情况下，获取最佳数据库参数配置。本功能不强制与数据库环境部署到一起，支持独立部署，脱离数据库安装环境独立运行。 + +## 客户价值 + +该工具可以在任意场景下，快速给出当前负载的调参配置，减少DBA的人工干预，提升运维效果，满足客户期望。 + +## 特性描述 + +调优程序X-Tuner包含三种运行模式，分别是: + +- recommend：通过用户指定的用户名等信息登录到数据库环境中，获取当前正在运行的workload特征信息，根据上述特征信息生成参数推荐报告。报告当前数据库中不合理的参数配置和潜在风险等；输出根据当前正在运行的workload行为和特征；输出推荐的参数配置。**该模式是秒级的，不涉及数据库的重启操作，其他模式可能需要反复重启数据库**。 +- train：通过用户提供的benchmark信息，不断地进行参数修改和benchmark的执行。通过反复的迭代过程，训练强化学习模型，以便用户在后面通过tune模式加载该模型进行调优。 +- tune：使用优化算法进行数据库参数的调优，当前支持两大类算法，一种是深度强化学习，另一种是全局搜索算法（全局优化算法）。深度强化学习模式要求先运行train模式，生成训练后的调优模型，而使用全局搜索算法则不需要提前进行训练，可以直接进行搜索调优。 + +## 特性增强 + +无。 + +## 特性约束 + +- 数据库状态正常、客户端能够正常连接、且要求数据库内导入数据，以便调优程序可以执行benchmark测试调优效果。 +- 使用本工具需要指定登录到数据库的用户身份，要求该登录到数据库上的用户具有足够的权限，以便可以获得充足的数据库状态信息。 +- 使用登录到数据库宿主机上的Linux用户，需要将$**GAUSSHOME/bin**添加到PATH环境变量中，即能够直接运行gsql、gs_guc、gs_ctl等数据库运维工具。 +- Python版本建议为Python3.6及以上，且运行环境中已经安装相应依赖，并能够正常启动调优程序。您可以独立安装一个python3.6+的环境，无需设置到全局环境变量中。不建议使用root用户权限安装本工具，如果以root身份安装本完毕工具，使用其他用户身份运行本工具时，需要确保配置文件有读取权限。 +- 本工具支持以三种模式运行，其中tune和train模式要求用户配置好benchmark运行环境，并导入数据，本工具将会通过迭代运行benchmark来判断修改后的参数是否有性能提升。 +- recommend模式建议在数据库正在执行workload的过程中执行，以便获得更准确的实时workload信息。 +- 本工具默认带有TPC-C、TPC-H、TPC-DS以及sysbench的benchmark运行脚本样例，如果用户使用上述benchmark对数据库系统进行压力测试，则可以对上述配置文件进行适度修改或配置。如果需要适配用户自己的业务场景，需要您参照benchmark目录中的template.py文件编写驱动您自定义benchmark的脚本文件。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/ai-capabilities/3-sqldiag-slow-sql-discovery.md b/product/zh/docs-mogdb/v2.1/characteristic-description/ai-capabilities/3-sqldiag-slow-sql-discovery.md new file mode 100644 index 0000000000000000000000000000000000000000..1dec2d7302af862d9e22bae2fb59afb60c4dc7b4 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/ai-capabilities/3-sqldiag-slow-sql-discovery.md @@ -0,0 +1,39 @@ +--- +title: SQLdiag：慢SQL发现 +summary: SQLdiag：慢SQL发现 +author: Guo Huan +date: 2022-05-07 +--- + +# SQLdiag：慢SQL发现 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +SQLdiag是一个SQL语句执行时间预测工具，通过模板化方法，实现在不获取SQL语句执行计划的前提下，依据语句逻辑相似度与历史执行记录，预测SQL语句的执行时间。 + +## 客户价值 + +- 工具不需要用户提供SQL执行计划，对数据库性能不会有任何影响。 +- 不同于业内其他算法只局限于OLAP或者OLTP，本工具场景更加广泛。 + +## 特性描述 + +SQLdiag着眼于数据库的历史SQL语句，通过对历史SQL语句的执行表现进行总结归纳，将之再用于推断新的未知业务上。由于短时间内数据库SQL语句执行时长不会有太大的差距，SQLdiag可以从历史数据中检测出与已执行SQL语句相似的语句结果集，并基于SQL向量化技术和模板化方法预测SQL语句执行时长。 + +## 特性增强 + +无。 + +## 特性约束 + +- 需要保证用户提供的历史日志及待预测负载的格式符合要求，可以使用数据库GUC参数开启收集，也可以通过监控工具采集。 +- 为保证预测准确率，用户提供的历史语句日志应尽可能全面并具有代表性。 +- 按照要求配置python环境。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/ai-capabilities/4-anomaly-detection-database-indicator-collection-forecasting-and-exception-monitoring.md b/product/zh/docs-mogdb/v2.1/characteristic-description/ai-capabilities/4-anomaly-detection-database-indicator-collection-forecasting-and-exception-monitoring.md new file mode 100644 index 0000000000000000000000000000000000000000..9c228b6d9bd95ec91c315f2b98c2381b4c9b69ab --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/ai-capabilities/4-anomaly-detection-database-indicator-collection-forecasting-and-exception-monitoring.md @@ -0,0 +1,44 @@ +--- +title: Anomaly-detection：数据库指标采集、预测与异常监控 +summary: Anomaly-detection：数据库指标采集、预测与异常监控 +author: Guo Huan +date: 2022-05-07 +--- + +# Anomaly-detection：数据库指标采集、预测与异常监控 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +anomaly_detection是MogDB集成的、可以用于数据库指标采集、预测以及异常监控与诊断的AI工具，是dbmind套间中的一个组件。支持采集的信息包括IO_Read、IO_Write、CPU_Usage、Memory_Usage以及数据库所占磁盘空间等。anomaly_detection可以同时监控多个指标，并预测每个指标未来的变化趋势，当发现某个指标在未来某段时间或者某个时刻会超出人工设置的阈值，该工具会通过日志进行报警。 + +## 客户价值 + +- 极大简化运维人员工作，释放大量劳动力，为公司节省成本。 +- 为用户提前发现异常情况，防止数据库发生意外，导致更大的损失。 + +## 特性描述 + +anomaly_detection由agent和detector两部分组成。agent和MogDB数据库环境部署在同一个服务器上，agent模块主要有两个作用。一个是定时采集数据库指标数据，并将采集到的数据存放到缓冲队列中；另一个作用是将缓冲队列中数据定时发送到detector端。 + +detector模块基于http或https和agent模块通信，因此它可以部署到任何可以与agent端进行通信的服务器上，该模块主要主要有两个作用。一个是接受agent端发送的数据，并将收集到的数据缓存在本地；另外一个作用是基于收集到的数据库指标数据，对该指标的未来变化趋势进行预测和异常报警。 + +## 特性增强 + +无。 + +## 特性约束 + +- 数据库状态正常，并且用户已将数据目录写入环境变量，并以PGDATA命名。 +- 使用登录到数据库宿主机上的Linux用户，需要将$**GAUSSHOME/bin**添加到PATH环境变量中，即能够直接运行gsql、gs_guc、gs_ctl等数据库运维工具。 +- Python版本建议为Python3.6及以上，且运行环境中已经安装相应依赖，并能够正常启动调优程序。 +- 本工具由Agent和Detector组成，Agent和Detector之间通过'http'或者'https'方式传递数据，因此需要保证Agent服务器和Detector服务器之间能够正常通信。 +- Detector模块运行server和monitor服务， 需要分别启动。 +- 如果使用'https'方式进行通信，需要准备CA证书以及Agent和Detector的证书和密钥，并分别放入项目根目录certificate下的ca、agent、server中，同时将密钥加密密码放入certificate的pwf中，并将其权限设置为600，防止其他用户进行读写操作。用户也可以使用share中的脚本生成证书和密钥。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/ai-capabilities/5-index-advisor-index-recommendation.md b/product/zh/docs-mogdb/v2.1/characteristic-description/ai-capabilities/5-index-advisor-index-recommendation.md new file mode 100644 index 0000000000000000000000000000000000000000..21d6e2e784595b9042d30e70fa1f4588cd375957 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/ai-capabilities/5-index-advisor-index-recommendation.md @@ -0,0 +1,40 @@ +--- +title: Index-advisor：索引推荐 +summary: Index-advisor：索引推荐 +author: Guo Huan +date: 2022-05-07 +--- + +# Index-advisor：索引推荐 + +## 可获得性 + +本特性自MogDB1.1.0开始引入。 + +## 特性简介 + +Index-advisor工具是一个覆盖多种任务级别和使用场景的数据库智能索引推荐工具，其具备单Query索引推荐功能、虚拟索引功能、workload级别索引推荐功能，可以为用户提供可靠的索引建议。 + +## 客户价值 + +为客户提供快速可靠的索引推荐功能，极大简化了运维人员的工作。 + +## 特性描述 + +单query索引推荐功能支持用户在数据库中直接进行操作，本功能基于查询语句的语义信息和数据库的统计信息，对用户输入的单条查询语句生成推荐的索引；虚拟索引功能支持用户在数据库中直接进行操作，本功能将模拟真实索引的建立，避免真实索引创建所需的时间和空间开销，用户基于虚拟索引，可通过优化器评估该索引对指定查询语句的代价影响；对于workload级别的索引推荐，用户可通过运行数据库外的脚本使用此功能，本功能将包含有多条DML语句的workload作为输入，最终生成一批可对整体workload的执行表现进行优化的索引。 + +## 特性增强 + +无。 + +## 特性约束 + +数据库状态正常、客户端能够正常连接。 + +当前执行用户下安装有gsql工具，该工具路径已被加入到PATH环境变量中。 + +具备Python3.6+的环境。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/ai-capabilities/6-deepsql-ai-algorithm-in-the-database.md b/product/zh/docs-mogdb/v2.1/characteristic-description/ai-capabilities/6-deepsql-ai-algorithm-in-the-database.md new file mode 100644 index 0000000000000000000000000000000000000000..c5c94908bf37b26bdc862a3d1ba18d3f7ea008e3 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/ai-capabilities/6-deepsql-ai-algorithm-in-the-database.md @@ -0,0 +1,38 @@ +--- +title: DeepSQL：库内AI算法 +summary: DeepSQL：库内AI算法 +author: Guo Huan +date: 2022-05-07 +--- + +# DeepSQL：库内AI算法 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +数据库DeepSQL特性实现DB4AI功能，即在数据库内实现AI算法，以更好的支撑大数据的快速分析和计算。这里提供了一整套基于SQL的机器学习、数据挖掘以及统计学的算法，用户可以直接使用SQL进行机器学习工作。Deep SQL能够抽象出端到端从数据到模型的研发过程，配合底层的引擎及自动优化，具备基础SQL知识的技术人员即可完成大部分的机器学习模型训练及预测任务。整个分析和处理都运行在数据库引擎中，用户可以直接分析和处理数据库内的数据，不需要在数据库和其它平台之间进行数据传递，避免在多个环境之间进行不必要地数据移动。 + +## 客户价值 + +客户可以在数据库内部直接使用AI算法，避免了大量数据的迁移带来的额外代价，同时模型可以由数据库统一管理，使用十分方便。 + +## 特性描述 + +DeepSQL是对MogDB DB4AI能力的增强。DeepSQL将常用的机器学习算法封装为UDF，支持60多个常用算法。其中包括回归算法（例如线性回归，逻辑回归，随机森林等）、分类算法（比如KNN等）、聚类算法（比如K-means）等。除了基础的机器学习算法之外，还包括图相关的算法，比如最短路径，图形直径等等算法；此外还支持数据处理（比如PCA），稀疏向量，统计学常用算法（比如协方差，Pearson系数计算等），训练集测试集分割方法，交叉验证方法等。 + +## 特性增强 + +无。 + +## 特性约束 + +- 环境中安装python2.7.12以上版本Python。 +- 数据库需要开启对PL/Python存储过程的支持。 +- 安装算法库需要拥有管理员权限的用户。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/application-development-interfaces/1-standard-sql.md b/product/zh/docs-mogdb/v2.1/characteristic-description/application-development-interfaces/1-standard-sql.md new file mode 100644 index 0000000000000000000000000000000000000000..dcd55cd801abd47d5f23adc6b96f3625823b540d --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/application-development-interfaces/1-standard-sql.md @@ -0,0 +1,38 @@ +--- +title: 支持标准SQL +summary: 支持标准SQL +author: Guo Huan +date: 2022-05-07 +--- + +# 支持标准SQL + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +SQL是用于访问和处理数据库的标准计算机语言。SQL标准的定义分成核心特性以及可选特性，绝大部分的数据库都没有100%支撑SQL标准。 + +MogDB数据库支持SQL:2011大部分的核心特性，同时还支持部分的可选特性，为使用者提供统一的SQL界面。 + +## 客户价值 + +标准SQL的引入为所有的数据库厂商提供统一的SQL界面，减少使用者的学习成本和应用程序的迁移代价。 + +## 特性描述 + +具体的特性列表请参见《参考指南》中“SQL语法”章节。 + +## 特性增强 + +无。 + +## 特性约束 + +无。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/application-development-interfaces/2-standard-development-interfaces.md b/product/zh/docs-mogdb/v2.1/characteristic-description/application-development-interfaces/2-standard-development-interfaces.md new file mode 100644 index 0000000000000000000000000000000000000000..6b59b1dbcba296041ec2b373729118a4a478744d --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/application-development-interfaces/2-standard-development-interfaces.md @@ -0,0 +1,36 @@ +--- +title: 支持标准开发接口 +summary: 支持标准开发接口 +author: Guo Huan +date: 2022-05-07 +--- + +# 支持标准开发接口 + +## 可获得性 + +本特性自MogDB 1.1.0 版本开始引入。 + +## 特性简介 + +支持ODBC 3.5及JDBC 4.0标准接口。 + +## 客户价值 + +提供业界标准的ODBC及JDBC接口，保证用户业务快速迁移至MogDB。 + +## 特性描述 + +目前支持标准的ODBC 3.5及JDBC 4.0接口，其中ODBC支持SUSE、Win32、Win64平台，JDBC无平台差异。 + +## 特性增强 + +增加JDBC对接第三方日志框架功能。JDBC对接第三方日志框架功能可满足用户对日志管控的需求。 + +## 特性约束 + +无。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/application-development-interfaces/3-postgresql-api-compatibility.md b/product/zh/docs-mogdb/v2.1/characteristic-description/application-development-interfaces/3-postgresql-api-compatibility.md new file mode 100644 index 0000000000000000000000000000000000000000..72d61da13c024ddf5f5c40d3629b5dae914a4414 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/application-development-interfaces/3-postgresql-api-compatibility.md @@ -0,0 +1,36 @@ +--- +title: PG接口兼容 +summary: PG接口兼容 +author: Guo Huan +date: 2022-05-07 +--- + +# PG接口兼容 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +兼容PSQL客户端，兼容PostgreSQL标准接口。 + +## 客户价值 + +兼容PSQL客户端，兼容PostgreSQL标准接口，能够与PG生态工具无缝对接。 + +## 特性描述 + +兼容PSQL客户端，兼容PostgreSQL标准接口。 + +## 特性增强 + +无。 + +## 特性约束 + +无。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/application-development-interfaces/4-pl-java.md b/product/zh/docs-mogdb/v2.1/characteristic-description/application-development-interfaces/4-pl-java.md new file mode 100644 index 0000000000000000000000000000000000000000..e08f4a9565756a3fa2455a63f592c43c204daf0f --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/application-development-interfaces/4-pl-java.md @@ -0,0 +1,66 @@ +--- +title: 支持PL/Java +summary: 支持PL/Java +author: Guo Huan +date: 2022-05-07 +--- + +# 支持PL/Java + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +支持java UDF。 + +## 客户价值 + +提供多种函数的开发环境。 + +## 特性描述 + +使用MogDB数据库的PL/Java函数，用户可以使用自己喜欢的Java IDE编写Java方法，并将包含这些方法的jar文件安装到MogDB数据库中，然后使用该方法。MogDB PL/Java基于开源tada PL/Java 1.5.2开发，PL/Java所使用的JDK版本为Huawei JDK V100R001C00SPC190B003-b09。 + +## 特性增强 + +无。 + +## 特性约束 + +- Java UDF可以实现一些较为简单的java计算，强烈建议不要在Java UDF中封装业务。 + +- 强烈建议不要在Java函数中使用任何方式连接数据库，包括但不限于JDBC。 + +- 建议用户使用Huawei JDK V100R001C00SPC190B003-b09编译Java方法和jar文件。 + +- 暂不支持的数据类型：除表1提及之外的数据类型，包括自定义类型，复杂数据类型（Java Array类及派生类）。 + +- 暂不支持UDAF、UDTF。 + + **表 1** PL/Java默认数据类型映射关系 + + | MogDB | **Java** | + | :---------- | :------------------------------------------------- | + | BOOLEAN | boolean | + | “char” | byte | + | bytea | byte[] | + | SMALLINT | short | + | INTEGER | int | + | BIGINT | long | + | FLOAT4 | float | + | FLOAT8 | double | + | CHAR | java.lang.String | + | VARCHAR | java.lang.String | + | TEXT | java.lang.String | + | name | java.lang.String | + | DATE | java.sql.Timestamp | + | TIME | java.sql.Time (stored value treated as local time) | + | TIMETZ | java.sql.Time | + | TIMESTAMP | java.sql.Timestamp | + | TIMESTAMPTZ | java.sql.Timestamp | + +## 依赖关系 + +PL/Java依赖JDK环境，目前MogDB中已包含JDK环境，无需用户安装。如果用户已安装（相同或不同版本的）JDK，也不会引起冲突，MogDB会使用Huawei JDK V100R001C00SPC190B003-b09来运行PL/Java。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/1-access-control-model.md b/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/1-access-control-model.md new file mode 100644 index 0000000000000000000000000000000000000000..ed3b7105b3a0007dde937ddbd164633b5b6f38a6 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/1-access-control-model.md @@ -0,0 +1,41 @@ +--- +title: 访问控制模型 +summary: 访问控制模型 +author: Guo Huan +date: 2022-05-07 +--- + +# 访问控制模型 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +管理用户访问权限，为用户分配完成任务所需要的最小权限。 + +## 客户价值 + +客户依据自身需求创建对应的数据库用户并赋予相应的权限给操作人员，将数据库使用风险降到最低。 + +## 特性描述 + +数据库提供了基于角色的访问控制模型和基于三权分立的访问控制模型。在基于角色的访问控制模型下，数据库用户可分为系统管理员用户、监控管理员用户、运维管理员用户、安全策略管理员用户以及普通用户。系统管理员创建角色或者用户组，并为角色分配对应的权限；监控管理员查看dbe_perf模式下的监控视图或函数；运维管理员使用Roach工具执行数据库备份恢复操作；安全策略管理员创建资源标签、脱敏策略、统一审计策略。用户通过绑定不同的角色获得角色所拥有的对应的操作权限。 + +在基于三权分立的访问控制模型下，数据库用户可分为系统管理员、安全管理员、审计管理员、监控管理员用户、运维管理员用户、安全策略管理员用户以及普通用户。安全管理员负责创建用户，系统管理员负责为用户赋权，审计管理员负责审计所有用户的行为。 + +默认情况下，使用基于角色的访问控制模型。客户可通过设置GUC参数enableSeparationOfDuty为on来切换。 + +## 特性增强 + +无。 + +## 特性约束 + +- 系统管理员的具体权限受GUC参数enableSeparationOfDuty控制。 +- 三权分立开关和关闭切换时需要重启数据库，且无法对新模型下不合理的用户权限进行自主识别，需要DBA识别并修正。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/10-row-level-access-control.md b/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/10-row-level-access-control.md new file mode 100644 index 0000000000000000000000000000000000000000..644588863adff724ee73f1d7032a763f678c106f --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/10-row-level-access-control.md @@ -0,0 +1,43 @@ +--- +title: 行级访问控制 +summary: 行级访问控制 +author: Guo Huan +date: 2022-05-07 +--- + +# 行级访问控制 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +行级访问控制特性将数据库访问控制精确到数据表行级别，使数据库达到行级访问控制的能力。不同用户执行相同的SQL查询操作，读取到的结果是不同的。 + +## 客户价值 + +不同用户执行相同的SQL查询操作，读取到的结果是不同的。 + +## 特性描述 + +用户可以在数据表创建行访问控制（Row Level Security）策略，该策略是指针对特定数据库用户、特定SQL操作生效的表达式。当数据库用户对数据表访问时，若SQL满足数据表特定的Row Level Security策略，在查询优化阶段将满足条件的表达式，按照属性（PERMISSIVE | RESTRICTIVE）类型，通过AND或OR方式拼接，应用到执行计划上。 + +行级访问控制的目的是控制表中行级数据可见性，通过在数据表上预定义Filter，在查询优化阶段将满足条件的表达式应用到执行计划上，影响最终的执行结果。当前受影响的SQL语句包括SELECT，UPDATE，DELETE。 + +## 特性增强 + +无。 + +## 特性约束 + +- 行级访问控制策略仅可以应用到SELECT、UPDATE和DELETE操作，不支持应用到INSERT和MERGE操作。 +- 支持对行存表、行存分区表、列存表、列存分区表、复制表、unlogged表、hash表定义行级访问控制策略，不支持HDFS表、外表、临时表定义行级访问控制策略。 +- 不支持对视图定义行级访问控制策略。 +- 同一张表上可以创建多个行级访问控制策略，一张表最多允许创建100个行级访问控制策略。 +- 初始用户和系统管理员不受行级访问控制策略的影响。 +- 对于设置了行级访问控制策略的表，需要谨慎授予其他用户对该表的trigger权限，以免其他用户利用触发器绕过行级访问控制策略。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/11-password-strength-verification.md b/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/11-password-strength-verification.md new file mode 100644 index 0000000000000000000000000000000000000000..1bf5f4927c8231868ff095b15eeaa1faf3a5c0e4 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/11-password-strength-verification.md @@ -0,0 +1,71 @@ +--- +title: 用户口令强度校验机制 +summary: 用户口令强度校验机制 +author: Guo Huan +date: 2022-05-07 +--- + +# 用户口令强度校验机制 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +对用户访问数据库所设置的口令强度进行校验。 + +## 客户价值 + +用户无法设置过低强度的口令，加固客户数据安全。 + +## 特性描述 + +初始化数据库、创建用户、修改用户时需要指定密码。密码必须满足强度校验，否则会提示用户重新输入密码。账户密码复杂度要求如下： + +- 包含大写字母（A-Z）的最少个数（password_min_uppercase） +- 包含小写字母（a-z）的最少个数（password_min_lowercase） +- 包含数字（0-9）的最少个数（password_min_digital） +- 包含特殊字符的最少个数（password_min_special） +- 密码的最小长度（password_min_length） +- 密码的最大长度（password_max_length） +- 至少包含上述四类字符中的三类。 +- 不能和用户名、用户名倒写相同，本要求为非大小写敏感。 +- 不能和当前密码、当前密码的倒写相同。 +- 不能是弱口令。 + +> ![img](https://cdn-mogdb.enmotech.com/docs-media/icon/icon-note.gif) **说明：** 弱口令指的是强度较低，容易被破解的密码，对于不同的用户或群体，弱口令的定义可能会有所区别，用户需要自己添加定制化的弱口令。 + +参数password_policy设置为1时表示采用密码复杂度校验，默认值为1。 + +弱口令字典中的口令存放在gs_global_config系统表中（name字段为weak_password的记录是储存的弱口令），当创建用户、修改用户需要设置密码时，将会把用户设置口令和弱口令字典中存放的口令进行对比，如果命中，则会提示用户该口令为弱口令，设置密码失败。 + +弱口令字典默认为空，用户通过以下语法可以对弱口令字典进行增加和删除，示例如下： + +``` +CREATE WEAK PASSWORD DICTIONARY WITH VALUES ('password1'), ('password2'); +DROP WEAK PASSWORD DICTIONARY; +``` + +其中“password1”，“password2”是用户事先准备的弱口令，该语句执行成功后将会存入弱口令系统表中。 + +当用户尝试通过CREATE WEAK PASSWORD DICTIONARY 插入表中已存在的弱口令时，会只在表中保留一条该弱口令。 + +DROP WEAK PASSWORD DICTIONARY语句会清空整张系统表弱口令相关部分。 + +gs_global_config系统表没有唯一索引，不建议用户通过COPY FROM命令重复用相同数据对该表进行操作。 + +若用户需要对弱口令相关操作进行审计，应设置audit_system_object参数中的第三位为1。 + +## 特性增强 + +MogDB 1.1.0版本实现了弱口令字典功能。 + +## 特性约束 + +- 初始用户、系统管理员和安全管理员可以查看、新增、删除弱口令字典。 +- 普通用户可以查看但是不能新增或者删除弱口令字典。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/12-equality-query-in-a-fully-encrypted-database.md b/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/12-equality-query-in-a-fully-encrypted-database.md new file mode 100644 index 0000000000000000000000000000000000000000..7ddfadf6abc085358544dca41ea2991e978e35c5 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/12-equality-query-in-a-fully-encrypted-database.md @@ -0,0 +1,110 @@ +--- +title: 全密态数据库等值查询 +summary: 全密态数据库等值查询 +author: Guo Huan +date: 2022-05-07 +--- + +# 全密态数据库等值查询 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +密态数据库意在解决数据全生命周期的隐私保护问题，使得系统无论在何种业务场景和环境下，数据在传输、运算以及存储的各个环节始终都处于密文状态。当数据拥有者在客户端完成数据加密并发送给服务端后，在攻击者借助系统脆弱点窃取用户数据的状态下仍然无法获得有效的价值信息，从而起到保护数据隐私的作用。 + +## 客户价值 + +由于整个业务数据流在数据处理过程中都是以密文形态存在，通过全密态数据库，可以实现： + +1. 保护数据在云上全生命周期的隐私安全，无论数据处于何种状态，攻击者都无法从数据库服务端获取有效信息。 +2. 帮助云服务提供商获取第三方信任，无论是企业服务场景下的业务管理员、运维管理员，还是消费者云业务下的应用开发者，用户通过将密钥掌握在自己手上，使得高权限用户无法获取数据有效信息。 +3. 让云数据库借助全密态能力更好的遵守个人隐私保护方面的法律法规。 + +## 特性描述 + +从用户视角来看，整个密态等值查询的主要功能分为三部分，主要通过新增的KeyTool工具以及MogDB的增强gsql客户端来实现。 + +首先是客户端密钥管理功能，用户借助新增的KeyTool工具来实现CMK的生成、销毁和更新，同时支持密钥导入导出。通过该KeyTool工具的导入导出功能，CMK可以在不同的客户端间进行传输；同时，KeyTool实现了单个客户端侧的密钥管理，通过配置管理文件，可以对密钥的存储、更新进行管理。 + +其次该特性提供密钥创建功能和加密表创建功能，通过新增SQL语法“CREATE CLINET MASTER KEY”和“CREATE COLUMN ENCRYPTION KEY”实现在数据库侧记录和管理CMK和CEK元信息，CMK和CEK信息被记录在新增的系统表中。通过扩展“CREATE TABLE”语法为表的每一个敏感信息列指定列加密密钥和加密算法，方便后续数据密文存储。 + +最后该特性对用户提供密态等值查询能力，该部分功能是整个特性的核心，虽然用户对整个密文查询无感知，但是在指定数据的加密信息后，针对该敏感数据的查询将受限于当前密态等值查询的支持规格。 + +从整体视角来看，该特性所实现的功能主要是为了用户基于敏感数据保护的诉求来存储管理数据并基于密文数据信息实现查询任务。 + +## 特性增强 + +无。 + +## 特性约束 + +- 数据以列级别进行加密，而无法按照行级别区分加密策略。 + +- 除Rename操作外，不支持通过Alter Table语法实现对加密表列的更改（包括加密列和非加密列之间的互转换），支持添加（Add）和删除（Drop）对应的加密列。 + +- 不支持对加密列设置大部分check限制性语法，但是支持check(column is not null)语法。 + +- 当support_extended_features = off时，不支持对加密列使用primary key、unique。当support_extended_features = on时，仅支持确定性加密列使用primary key、unique。 + +- 不支持不同数据类型之间的隐式转换。 + +- 不支持不同数据类型密文间的集合操作。 + +- 不支持加密列创建范围分区。 + +- 加密列仅支持repeat和empty_blob()函数。 + +- 当前版本只支持gsql和JDBC（部署在linux操作系统）客户端，暂不支持ODBC等其他客户端实现密态等值查询。 + +- 使用JDBC客户端时，密态等值查询特性不能与负载均衡或自动选主特性同时开启。 + +- 只支持通过客户端执行copy from stdin的方式、\copy命令的方式以及insert into values(…)的方式往密态表中导入数据。 + +- 不支持从加密表到文件之间的copy操作。 + +- 不支持包括排序、范围查询以及模糊查询等在内的除等值以外的各种密态查询。 + +- 支持部分函数存储过程密态语法，密态支持函数存储过程具体约束查看开发者指南密态支持函数/存储过程章节。 + +- 不支持通过insert into…select…，merge into语法将非加密表数据插入到加密表数据中。 + +- 对于处于连接状态的连接请求，只有触发更新缓存的操作（更改用户，解密加密列失败等）和重新建连后才能感知服务端CEK信息变更。 + +- 不支持在由随机加密算法加密的列上进行密态等值查询。 + +- 对于密态等值查询运算中如果参与比较的两个属性条件采用不同的数据加密密钥，返回报错。 + +- 密态等值查询不支持时序表、外表，不支持ustore存储引擎加密。 + +- 对于数据库服务侧配置变更（pg_settings系统表、权限、密钥和加密列等信息）, 需要重新建立JDBC连接保证配置变更生效。 + +- 不支持多条SQL语句一起执行，insert into语句多批次执行场景不受此条约束限制。 + +- 密态等值查询支持的数据类型包括： + + | 数据类 | 类型 | 描述 | + | ---------- | ------------------ | ----------------------------------- | + | 整型 | tinyint/tinyint(n) | 微整数，同int1 | + | | smallint | 小整数，同int2 | + | | int4 | 常用整数 | + | | binary_integer | Oracle兼容类型，常用整数 | + | | bigint/bigint(n) | 大整数，同int8 | + | 数值类型 | numeric(p,s) | 精度为p的准确数值类型 | + | | number | Oracle兼容类型，等同numeric(p,s) | + | 浮点类型 | float4 | 单精度浮点数 | + | | float8 | 双精度浮点数 | + | | double precision | 双精度浮点数 | + | 字符类型 | char/char(n) | 定长字符串，不足补空格，默认精度为1 | + | | varchar(n) | 变长字符串，n是指允许的最大字节长度 | + | | text | 文本类型 | + | | varchar2(n) | Oracle兼容类型，等同varchar(n) | + | | clob | 大文本类型 | + | 二进制类型 | bytea | 变长的二进制字符串 | + | | blob | 二进制大对象 | + +## 依赖关系 + +无。 diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/13-ledger-database-mechanism.md b/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/13-ledger-database-mechanism.md new file mode 100644 index 0000000000000000000000000000000000000000..0b6bc7eb4ccf05e32bd63d9ef34493228680bd55 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/13-ledger-database-mechanism.md @@ -0,0 +1,43 @@ +--- +title: 账本数据库机制 +summary: 账本数据库机制 +author: Guo Huan +date: 2022-05-07 +--- + +# 账本数据库机制 + +## 可获得性 + +本特性自MogDB 2.1.0版本开始引入。 + +## 特性简介 + +账本数据库特性，对用户指定的防篡改表增加校验信息，并记录用户对其数据的操作历史，通过数据和操作历史的一致性校验来保证用户数据无法被恶意篡改。在用户对防篡改表执行DML操作时，系统对防篡改表增加少量额外的行级校验信息，同时记录操作的SQL语句和数据的变化历史。通过特性提供的校验接口，用户可以方便的校验防篡改表中的数据与系统记录的操作信息是否一致。 + +## 客户价值 + +账本数据库通过提供对用户数据的操作记录、数据历史变化记录以及易用的一致性的校验接口，方便用户随时校验数据库中的敏感信息是否发生恶意篡改，有效提高数据库防篡改能力。 + +## 特性描述 + +账本数据库采用账本Schema对普通表和防篡改用户表进行隔离。用户在账本Schema中创建的行存表具有防篡改属性，即为防篡改用户表。用户向防篡改用户表中插入数据时，系统会自动生成少量行级校验信息。在用户执行DML时，系统会在全局区块表（GS_GLOBAL_CHAIN）中记录用户的操作、在用户表相应的历史表中记录数据的更改等信息，操作记录、数据变化记录和用户表中的数据三者严格保持一致。账本数据库提供高性能校验接口，能够供用户方便的校验数据的一致性，如果一致性校验失败，则说明数据可能发生篡改，需要及时联系审计管理员回溯操作记录历史。 + +## 特性增强 + +无。 + +## 特性约束 + +- 防篡改模式下的行存表具有防篡改属性，而临时表、UNLOGGED表、列存表、时序表等均不具有防篡改属性。 +- 不允许修改防篡改用户表的结构，不允许truncate防篡改相关表，不允许将防篡改用户表切换到普通的Schema中，不允许将非防篡改表切换到防篡改Schema中。 +- 防篡改表如果为分区表，则不支持exchange partition、drop partition、truncate partition等操作。 +- 不支持使用函数、TRIGGER修改防篡改用户表数据。 +- 防篡改用户表创建时不能有名为“hash”的列。 +- 普通用户调用篡改校验接口只能校验自己有权查询的表。 +- 只允许审计管理员和初始用户查询全局区块表和BLOCKCHAIN模式中的表，普通用户无权访问，所有用户均无权修改。 +- 根据用户历史表命名规则，若待创建表的Schema或表名以'_'结尾或开头，可能会出现对应历史表名与已有表名冲突的情况，需要重新命名。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/2-separation-of-control-and-access-permissions.md b/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/2-separation-of-control-and-access-permissions.md new file mode 100644 index 0000000000000000000000000000000000000000..c8cd9df8e9d2ce4b295d874e004c732b7c726161 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/2-separation-of-control-and-access-permissions.md @@ -0,0 +1,46 @@ +--- +title: 控制权和访问权分离 +summary: 控制权和访问权分离 +author: Guo Huan +date: 2022-05-07 +--- + +# 控制权和访问权分离 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +支持控制权和访问权分离。 + +## 客户价值 + +针对管理员用户，表对象的控制权和访问权要能够分离，提高普通用户数据安全性。 + +## 特性描述 + +对于有多个业务部门，各部门间使用不同的数据库用户进行业务操作，同时有一个同级的数据库维护部门使用数据库管理员进行维护操作的场景下，业务部门可能希望在未经授权的情况下，管理员用户只能对各部门的数据进行控制操作（DROP、ALTER、TRUNCATE），但是不能进行访问操作（INSERT、DELETE、UPDATE、SELECT、COPY）。即针对管理员用户，表对象的控制权和访问权要能够分离，提高普通用户数据安全性。 + +三权分立情况下，管理员对其他用户放在属于各自模式下的表无权限。但是，这种无权限包含了无控制权限，因此不能满足上面的诉求。为此，MogDB提供了私有用户方案。即在非三权分立模式下，创建具有INDEPENDENT属性的私有用户。具备CREATEROLE权限或者是系统管理员权限的用户可以创建私有用户或者修改普通用户的属性为私有用户，普通用户也可以修改自己的属性为私有用户。 + +``` +MogDB=# CREATE USER user_independent WITH INDEPENDENT IDENTIFIED BY "1234@abc"; +``` + +针对该用户的表对象，系统管理员在未经其授权前，只能进行控制操作（DROP、ALTER、TRUNCATE），无权进行INSERT、DELETE、SELECT、UPDATE、COPY、GRANT、REVOKE、ALTER OWNER操作。 + +## 特性增强 + +无。 + +## 特性约束 + +对于私有用户的表，需要谨慎授予其他用户对该表的trigger权限，以免其他用户利用触发器查看私有用户的数据。 + +若将私有用户表的相关权限授予其他非私有用户，系统管理员也会获得同样的权限。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/3-database-encryption-authentication.md b/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/3-database-encryption-authentication.md new file mode 100644 index 0000000000000000000000000000000000000000..6c29e8a17a04ac317bfecd163877ca6505542f32 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/3-database-encryption-authentication.md @@ -0,0 +1,36 @@ +--- +title: 数据库认证机制 +summary: 数据库认证机制 +author: Guo Huan +date: 2022-05-07 +--- + +# 数据库认证机制 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +提供基于客户端/服务端（C/S）模式的客户端连接认证机制。 + +## 客户价值 + +加密认证过程中采用单向Hash不可逆加密算法PBKDF2，有效防止彩虹攻击。 + +## 特性描述 + +MogDB采用基本的客户端连接认证机制，客户端发起连接请求后，由服务端完成信息校验并依据校验结果发送认证所需信息给客户端（认证信息包括盐值，token以及服务端签名信息）。客户端响应请求发送认证信息给服务端，由服务端调用认证模块完成对客户端认证信息的认证。用户的密码被加密存储在内存中。整个过程中口令加密存储和传输。当用户下次登录时通过计算相应的hash值并与服务端存储的key值比较来进行正确性校验。 + +## 特性增强 + +统一加密认证过程中的消息处理流程，可有效防止攻击者通过抓取报文猜解用户名或者密码的正确性。 + +## 特性约束 + +无。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/4-data-encryption-and-storage.md b/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/4-data-encryption-and-storage.md new file mode 100644 index 0000000000000000000000000000000000000000..3a479894526fc3f59e657b9ec15337704383b59d --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/4-data-encryption-and-storage.md @@ -0,0 +1,44 @@ +--- +title: 数据加密存储 +summary: 数据加密存储 +author: Guo Huan +date: 2022-05-07 +--- + +# 数据加密存储 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +提供对导入数据的加密存储。 + +## 客户价值 + +为客户提供加密导入接口，对客户认为是敏感信息的数据进行加密后存储在表内。 + +## 特性描述 + +MogDB提供加密函数gs_encrypt_aes128()、gs_encrypt()和解密函数gs_decrypt_aes128()、gs_decrypt()接口。通过加密函数，可以对需要输入到表内的某列数据进行加密后再存储到表格内。调用格式为: + +**gs_encrypt_aes128(column, key),gs_encrypt(decryptstr,keystr,decrypttype)** + +其中key为用户指定的初始口令，用于派生加密密钥。当客户需要对整张表进行加密处理时，则需要为每一列单独书写加密函数。 + +当具有对应权限的用户需要查看具体的数据时，可通过解密函数接口对相应的属性列进行解密处理，调用格式为： + +**gs_decrypt_aes128(column, key),gs_decrypt(decryptstr,keystr,decrypttype)** + +## 特性增强 + +无。 + +## 特性约束 + +无。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/5-database-audit.md b/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/5-database-audit.md new file mode 100644 index 0000000000000000000000000000000000000000..792886a2cdccc1ea711f6836702e21ef42386ed4 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/5-database-audit.md @@ -0,0 +1,36 @@ +--- +title: 数据库审计 +summary: 数据库审计 +author: Guo Huan +date: 2022-05-07 +--- + +# 数据库审计 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +审计日志记录用户对数据库的启停、连接、DDL、DML、DCL等操作。 + +## 客户价值 + +审计日志机制主要增强数据库系统对非法操作的追溯及举证能力。 + +## 特性描述 + +数据库安全对数据库系统来说至关重要。MogDB将用户对数据库的所有操作写入审计日志。数据库安全管理员可以利用这些日志信息，重现导致数据库现状的一系列事件，找出非法操作的用户、时间和内容等。 + +## 特性增强 + +无。 + +## 特性约束 + +无。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/6-network-communication-security.md b/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/6-network-communication-security.md new file mode 100644 index 0000000000000000000000000000000000000000..b285a501943e0b94fa97a973d92d784a0767cb87 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/6-network-communication-security.md @@ -0,0 +1,44 @@ +--- +title: 网络通信安全 +summary: 网络通信安全 +author: Guo Huan +date: 2022-05-07 +--- + +# 网络通信安全 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +为保护敏感数据在Internet上传输的安全性，MogDB支持通过SSL加密客户端和服务器之间的通讯。 + +## 客户价值 + +保证客户的客户端与服务器通讯安全。 + +## 特性描述 + +MogDB支持SSL协议标准。SSL（Secure Socket Layer）协议是一种安全性更高的应用层通信协议，主要用于Web安全传输，SSL包含记录层和传输层，记录层协议确定传输层数据的封装格式，传输层安全协议使用X.509认证。SSL协议利用非对称加密演算来对通信方做身份认证，之后交换对称密钥作为会谈密钥。通过SSL协议可以有效保障两个应用间通信的保密性和可靠性，使客户与服务器之间的通信不被攻击者窃听。 + +MogDB支持TLS 1.2协议标准。TLS 1.2协议是一种安全性更高的传输层通信协议，它包括两个协议组，TLS记录协议和TLS握手协议，每一组协议具有很多不同格式的信息。TLS协议是独立于应用协议的，高层协议可以透明地分布在TLS协议上面。通过TLS协议可保证通信双方的数据保密性和数据完整性。 + +## 特性增强 + +证书签名算法强度检查：对于一些强度较低的签名算法，给出告警信息，提醒客户更换包含高强度签名算法的证书。 + +证书超时时间检查：如果距离超期日期小于7天则给出告警信息，提醒客户端更换证书。 + +证书权限检查：在创建连接阶段对证书的权限进行校验。 + +## 特性约束 + +从CA认证中心申请到正式的服务器、客户端的证书和密钥。（假设服务器的私钥为server.key，证书为server.crt，客户端的私钥为client.key，证书为client.crt，CA根证书名称为cacert.pem。） + +需要打开SSL开关，并且配置证书和连接方式。 + +## 依赖关系 + +OpenSSL \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/7-resource-label.md b/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/7-resource-label.md new file mode 100644 index 0000000000000000000000000000000000000000..32444d61ea88e7643364f73049e30adaf49c0402 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/7-resource-label.md @@ -0,0 +1,49 @@ +--- +title: 资源标签机制 +summary: 资源标签机制 +author: Guo Huan +date: 2022-05-07 +--- + +# 资源标签机制 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +数据库资源是指数据库所记录的各类对象，包括数据库、模式、表、列、视图、触发器等，数据库对象越多，数据库资源的分类管理就越繁琐。资源标签机制是一种通过对具有某类相同“特征”的数据库资源进行分类标记而实现资源分类管理的一种技术。当管理员对数据库内某些资源“打上”标签后，可以基于该标签进行如审计或数据脱敏的管理操作，从而实现对标签所包含的所有数据库资源进行安全管理。 + +## 客户价值 + +合理的制定资源标签能够有效的进行数据对象分类，提高对象管理效率，降低安全策略配置的复杂性。当管理员需要对某组数据库资源对象做统一审计或数据脱敏等安全管理动作时，可将这些资源划分到一个资源标签，该标签即包含了具有某类特征或需要统一配置某种策略的数据库资源，管理员可直接对资源标签执行管理操作，大大降低了策略配置的复杂性和信息冗余程度，提高了管理效率。 + +## 特性描述 + +资源标签机制是将当前数据库内包含的各种资源进行“有选择性的”分类，管理员可以使用如下SQL语法进行资源标签的创建，从而将一组数据库资源打上标签： + +``` +CREATE RESOURCE LABEL schm_lb ADD SCHEMA (schema_for_label); +CREATE RESOURCE LABEL tb_lb ADD TABLE (schema_for_label.table_for_label); +CREATE RESOURCE LABEL col_lb ADD COLUMN (schema_for_label.table_for_label.column_for_label); +CREATE RESOURCE LABEL multi_lb ADD SCHEMA (schema_for_label), TABLE (table_for_label); +``` + +其中，schema_for_label、table_for_label、column_for_label分别为待标记模式、表、列。schm_lb标签包含了模式schm_for_label；tb_lb包含了表table_for_label；col_lb包含了列column_for_label；multi_lb包含模式schm_for_label和列table_for_label。对这些已配置的资源标签进行如统一审计或动态数据脱敏也即是对标签所包含的每一个数据库资源进行管理。 + +当前，资源标签所支持的数据库资源类型包括：SCHEMA、TABLE、COLUMN、VIEW、FUNCTION。 + +## 特性增强 + +无。 + +## 特性约束 + +- 资源标签需要由具备POLADMIN和SYSADMIN属性的用户或初始用户创建。 +- 不支持对临时表创建资源标签。 +- 同一个基本表的列只可能属于一个资源标签。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/8-unified-audit.md b/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/8-unified-audit.md new file mode 100644 index 0000000000000000000000000000000000000000..b30f32efb8b75bdc47a2f3c9e06fc559dfa064d1 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/8-unified-audit.md @@ -0,0 +1,68 @@ +--- +title: 统一审计机制 +summary: 统一审计机制 +author: Guo Huan +date: 2022-05-07 +--- + +# 统一审计机制 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +审计机制是行之有效的安全管理方案，可有效解决攻击者抵赖，审计的范围越大，可监控的行为就越多，而产生的审计日志就越多，影响实际审计效率。统一审计机制是一种通过定制化审计策略实现高效安全审计管理的技术。当管理员定义审计对象和审计行为后，用户执行的任务如果关联到对应的审计策略，则生成对应的审计行为，并记录审计日志。定制化审计策略可涵盖常见的用户管理活动、DDL和DML行为，满足日常审计诉求。 + +## 客户价值 + +审计是日常安全管理中必不可少的行为，当使用传统审计机制审计某种行为时，如SELECT，会导致产生大量的审计日志，进而增加整个系统的I/O，影响系统的性能；另一方面，大量的审计日志会影响管理员的审计效率。统一审计机制使得客户可以定制化生成审计日志的策略，如只审计数据库账户A查询某个表的行为。通过定制化审计，可以大大减少生成审计日志的数量，从而在保障审计行为的同时降低对系统性能的影响。而定制化审计策略可以提升管理员的审计效率。 + +## 特性描述 + +统一审计机制基于资源标签进行审计行为定制化，且将当前所支持的审计行为划分为access类和privileges类。一个完整的审计策略创建的SQL语法如下所示： + +``` +CREATE RESOURCE LABEL auditlabel add table(table_for_audit1, table_for_audit2); + +CREATE AUDIT POLICY audit_select_policy ACCESS SELECT ON LABEL(auditlabel) FILTER ON ROLES(usera); + +CREATE AUDIT POLICY audit_admin_policy PRIVILEGES ALTER, DROP ON LABEL(auditlabel) FILTER ON IP(local); +``` + +其中，auditlabel为本轮计划审计的资源标签，该资源标签中包含了两个表对象；audit_select_policy定义了用户usera对auditlabel对象的SELECT行为的审计策略，不区分访问源；audit_admin_policy定义了从本地对auditlabel对象进行ALTER和DROP操作行为的审计策略，不区分执行用户；当不指定ACCESS和PRIVILEGES的具体行为时，表示审计针对某一资源标签的所有支持的DDL和DML行为。当不指定具体的审计对象时，表示审计针对所有对象的操作行为。统一审计策略的增删改也会记录在统一审计日志中。 + +当前，统一审计支持的审计行为包括： + +| SQL类型 | 支持操作和对象类型 | +| ------- | ------------------------------------------------------------ | +| **DDL** | 操作：ALL ALTER ANALYZE COMMENT CREATE DROP GRANT REVOKE
SET SHOW
对象：DATABASE SCHEMA FUNCTION TRIGGER TABLE SEQUENCE FOREIGN_SERVER FOREIGN_TABLE TABLESPACE ROLE/USER INDEX VIEW DATA_SOURCE | +| **DML** | 操作：ALL COPY DEALLOCATE DELETE EXECUTE REINDEX INSERT
PREPARE SELECT TRUNCATE UPDATE | + +## 特性增强 + +无。 + +## 特性约束 + +- 统一审计策略需要由具备POLADMIN或SYSADMIN属性的用户或初始用户创建，普通用户无访问安全策略系统表和系统视图的权限。 +- 统一审计策略语法要么针对DDL行为，要么针对DML语法行为，同一个审计策略不可同时包含DDL和DML行为；统一审计策略目前支持最多设置98个。 +- 统一审计监控用户通过客户端执行的SQL语句，而不会记录数据库内部SQL语句。 +- 同一个审计策略下，相同资源标签可以绑定不同的审计行为，相同行为可以绑定不同的资源标签， 操作”ALL”类型包括DDL或者DML下支持的所有操作。 +- 同一个资源标签可以关联不同的统一审计策略，统一审计会按照SQL语句匹配的策略依次打印审计信息。 +- 统一审计策略的审计日志单独记录，暂不提供可视化查询接口，整个日志依赖于操作系统自带rsyslog服务，通过配置完成日志归档。 +- 在云服务场景下，日志需要存储在OBS服务中；在混合云场景下，可部署Elastic Search进行日志收集和可视化处理。 +- FILTER中的APP项建议仅在同一信任域内使用，由于客户端不可避免的可能出现伪造名称的情况，该选项使用时需要与客户端联合形成一套安全机制，减少误用风险。一般情况下不建议使用，使用时需要注意客户端仿冒的风险。 +- FILTER中的IP地址以ipv4为例支持如下格式。 + +| ip地址格式 | 示例 | +| ---------- | ------------------------ | +| 单ip | 127.0.0.1 | +| 掩码表示ip | 127.0.0.1\|255.255.255.0 | +| cidr表示ip | 127.0.0.1/24 | +| ip区间 | 127.0.0.1-127.0.0.5 | + +## 依赖关系 + +在公有云服务场景下，依赖OSS服务或OBS服务存储日志。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/9-dynamic-data-anonymization.md b/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/9-dynamic-data-anonymization.md new file mode 100644 index 0000000000000000000000000000000000000000..6df341e7e2393df1084a6f7c6e5ea47b35677455 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/database-security/9-dynamic-data-anonymization.md @@ -0,0 +1,93 @@ +--- +title: 动态数据脱敏机制 +summary: 动态数据脱敏机制 +author: Guo Huan +date: 2022-05-07 +--- + +# 动态数据脱敏机制 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +数据脱敏是行之有效的数据库隐私保护方案之一，可以在一定程度上限制非授权用户对隐私数据的窥探。动态数据脱敏机制是一种通过定制化制定脱敏策略从而实现对隐私数据保护的一种技术，可以有效地在保留原始数据的前提下解决非授权用户对敏感信息的访问问题。当管理员指定待脱敏对象和定制数据脱敏策略后，用户所查询的数据库资源如果关联到对应的脱敏策略时，则会根据用户身份和脱敏策略进行数据脱敏，从而限制非授权用户对隐私数据的访问。 + +## 客户价值 + +数据隐私保护是数据库安全所需要具备的安全能力之一，可以在一定程度上限制非授权用户对隐私数据的访问，保证隐私数据安全。动态数据脱敏机制可以通过配置脱敏策略实现对指定数据库资源信息的隐私保护，另一方面，脱敏策略的配置也具有一定的灵活性，可以仅针对特定用户场景实现有针对性的隐私保护能力。 + +## 特性描述 + +动态数据脱敏机制基于资源标签进行脱敏策略的定制化，可根据实际场景选择特定的脱敏方式，也可以针对某些特定用户制定脱敏策略。一个完整的脱敏策略创建的SQL语法如下所示： + +``` +CREATE RESOURCE LABEL label_for_creditcard ADD COLUMN(user1.table1.creditcard); +CREATE RESOURCE LABEL label_for_name ADD COLUMN(user1.table1.name); +CREATE MASKING POLICY msk_creditcard creditcardmasking ON LABEL(label_for_creditcard); +CREATE MASKING POLICY msk_name randommasking ON LABEL(label_for_name) FILTER ON IP(local), ROLES(dev); +``` + +其中，label_for_creditcard和msk_name为本轮计划脱敏的资源标签，分别包含了两个列对象；creditcardmasking、randommasking为预置的脱敏函数；msk_creditcard定义了所有用户对label_for_creditcard标签所包含的资源访问时做creditcardmasking的脱敏策略，不区分访问源；msk_name定义了本地用户dev对label_for_name标签所包含的资源访问时做randommasking的脱敏策略；当不指定FILTER对象时则表示对所有用户生效，否则仅对标识场景的用户生效。 + +当前，预置的脱敏函数包括： + +| 脱敏函数名 | 示例 | +| ----------------- | ------------------------------------------------------------ | +| creditcardmasking | '4880-9898-4545-2525' 将会被脱敏为 'xxxx-xxxx-xxxx-2525'，该函数仅对后4位之前的数字进行脱敏 | +| basicemailmasking | 'abcd@gmail.com' 将会被脱敏为'xxxx@gmail.com'， 对出现第一个'@'之前的文本进行脱敏 | +| fullemailmasking | 'abcd@gmail.com' 将会被脱敏为 'xxxx@xxxxx.com'，对出现最后一个'.'之前的文本（除'@'符外）进行脱敏 | +| alldigitsmasking | 'alex123alex' 将会被脱敏为 'alex000alex'， 仅对文本中的数字进行脱敏 | +| shufflemasking | 'hello word' 将会被随机打乱顺序脱敏为 'hlwoeor dl'， 该函数通过字符乱序排列的方式实现，属于弱脱敏函数，语义较强的字符串不建议使用该函数脱敏。 | +| randommasking | 'hello word' 将会被脱敏为 'ad5f5ghdf5'，将文本按字符随机脱敏 | +| maskall | '4880-9898-4545-2525' 将会被脱敏为 'xxxxxxxxxxxxxxxxxxx' | + +每个脱敏函数规格如下： + +| 脱敏函数名 | 支持的数据类型 | +| ----------------- | ------------------------------------------------------------ | +| creditcardmasking | BPCHAR, VARCHAR, NVARCHAR, TEXT（注：仅针对信用卡格式的文本类数据） | +| basicemailmasking | BPCHAR, VARCHAR, NVARCHAR, TEXT（注：仅针对email格式的文本类型数据） | +| fullemailmasking | BPCHAR, VARCHAR, NVARCHAR, TEXT（注：仅针对email格式的文本类型数据） | +| alldigitsmasking | BPCHAR, VARCHAR, NVARCHAR, TEXT（注：仅针对包含数字的文本类型数据） | +| shufflemasking | BPCHAR, VARCHAR, NVARCHAR, TEXT（注：仅针对文本类型数据） | +| randommasking | BPCHAR, VARCHAR, NVARCHAR, TEXT（注：仅针对文本类型数据） | +| maskall | BOOL, RELTIME, TIME, TIMETZ, INTERVAL, TIMESTAMP, TIMESTAMPTZ, SMALLDATETIME, ABSTIME,TEXT, BPCHAR, VARCHAR, NVARCHAR2, NAME, INT8, INT4, INT2, INT1, NUMRIC, FLOAT4, FLOAT8, CASH | + +对于不支持的数据类型，默认使用maskall函数进行数据脱敏，若数据类型不属于maskall支持的数据类型则全部使用数字0进行脱敏，如果脱敏列涉及隐式转换，则结果以隐式转换后的数据类型为基础进行脱敏。另外需要说明的是，如果脱敏策略应用到数据列并生效，此时对该列数据的操作将以脱敏后的结果为基础而进行。 + +动态数据脱敏适用于和实际业务紧密相关的场景，根据业务需要为用户提供合理的脱敏查询接口，以避免通过撞库而获取原始数据。 + +## 特性增强 + +无。 + +## 特性约束 + +- 动态数据脱敏策略需要由具备POLADMIN或SYSADMIN属性的用户或初始用户创建，普通用户没有访问安全策略系统表和系统视图的权限。 +- 动态数据脱敏只在配置了脱敏策略的数据表上生效，而审计日志不在脱敏策略的生效范围内。 +- 在一个脱敏策略中，对于同一个资源标签仅可指定一种脱敏方式，不可重复指定。 +- 不允许多个脱敏策略对同一个资源标签进行脱敏，除以下脱敏场景外：使用FILTER指定策略生效的用户场景，包含相同资源标签的脱敏策略间FILTER生效场景无交集，此时可以根据用户场景明确辨别资源标签被哪种策略脱敏。 +- Filter中的APP项建议仅在同一信任域内使用，由于客户端不可避免的可能出现伪造名称的情况，该选项使用时需要与客户端联合形成一套安全机制，减少误用风险。一般情况下不建议使用，使用时需要注意客户端仿冒的风险。 +- 对于带有query子句的INSERT或MERGE INTO操作，如果源表中包含脱敏列，则上述两种操作中插入或更新的结果为脱敏后的值，且不可还原。 +- 在内置安全策略开关开启的情况下，执行ALTER TABLE EXCHANGE PARTITION操作的源表若在脱敏列则执行失败。 +- 对于设置了动态数据脱敏策略的表，需要谨慎授予其他用户对该表的trigger权限，以免其他用户利用触发器绕过脱敏策略。 +- 最多支持创建98个动态数据脱敏策略。 +- 仅支持使用上述七种预置脱敏策略。 +- 仅支持对只包含COLUMN属性的资源标签做脱敏。 +- 仅支持对基本表的列进行数据脱敏。 +- 仅支持对SELECT查询到的数据进行脱敏。 +- FILTER中的IP地址以ipv4为例支持如下格式。 + +| ip地址格式 | 示例 | +| ---------- | ------------------------ | +| 单ip | 127.0.0.1 | +| 掩码表示ip | 127.0.0.1\|255.255.255.0 | +| cidr表示ip | 127.0.0.1/24 | +| ip区间 | 127.0.0.1-127.0.0.5 | + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/1-support-for-functions-and-stored-procedures.md b/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/1-support-for-functions-and-stored-procedures.md new file mode 100644 index 0000000000000000000000000000000000000000..7028f7248b5f5a8993614bbf95bce2bb0511bff7 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/1-support-for-functions-and-stored-procedures.md @@ -0,0 +1,38 @@ +--- +title: 函数及存储过程支持 +summary: 函数及存储过程支持 +author: Guo Huan +date: 2022-05-07 +--- + +# 函数及存储过程支持 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +函数和存储过程是数据库中的一种重要对象，主要功能将用户特定功能的SQL语句集进行封装，并方便调用。 + +## 客户价值 + +1. 允许客户模块化程序设计，对SQL语句集进行封装，调用方便。 +2. 存储过程会进行编译缓存，可以提升用户执行SQL语句集的速度。 +3. 系统管理员通过限制执行某一存储过程的权限，能够实现对相应的数据的访问权限的限制，避免了非授权用户对数据的访问，保证了数据的安全。 + +## 特性描述 + +MogDB支持SQL标准中的函数及存储过程，其中存储过程兼容了部分主流数据库存储过程的语法，增强了存储过程的易用性。 + +## 特性增强 + +支持PL/PGSQL单步调试功能。 + +## 特性约束 + +无。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/10-autonomous-transaction.md b/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/10-autonomous-transaction.md new file mode 100644 index 0000000000000000000000000000000000000000..262d4ec2bfdf4aa1dae6c6562d547fdb358157d6 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/10-autonomous-transaction.md @@ -0,0 +1,44 @@ +--- +title: 自治事务 +summary: 自治事务 +author: Guo Huan +date: 2022-05-07 +--- + +# 自治事务 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +子事务的提交不受主事务提交或回滚影响的一类事务。 + +## 客户价值 + +满足客户多样化使用场景。 + +## 特性描述 + +自治事务为在主事务执行过程中，以独立的事务上下文执行指定的某一类SQL语句，其commit和rollback不受主事务commit和rollback的影响。 + +用户自定义函数、存储过程支持自治事务。 + +典型使用场景是，用一个表记录主事务执行过程中的操作信息，并且在主事务失败回退的时候，表中记录的操作信息不予回退。 + +## 特性增强 + +无。 + +## 特性约束 + +- 触发器函数不支持自治事务。 +- 函数或者存储过程的自治事务块中，静态sql语句不支持变量传递。 +- 自治事务不支持执行嵌套。 +- 包含自治事务的函数，不支持参数传递的返回值。 +- 包含自治事务的存储过程/函数，不支持exception异常处理。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/11-global-temporary-table.md b/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/11-global-temporary-table.md new file mode 100644 index 0000000000000000000000000000000000000000..2ee1f1e6dbffb8f905174c12dd1bca8ed88b4e61 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/11-global-temporary-table.md @@ -0,0 +1,45 @@ +--- +title: 全局临时表 +summary: 全局临时表 +author: Guo Huan +date: 2022-05-07 +--- + +# 全局临时表 + +## 可获得性 + +本特性自MogDB 1.1.0 版本开始引入。 + +## 特性简介 + +临时表顾名思义是不保证持久化的表，其生命周期一般跟session或者事务绑定，可以方便用于表达处理过程中的一些临时数据存放，加速查询。 + +## 客户价值 + +提升临时表的表达能力和易用性。 + +## 特性描述 + +全局临时表的元数据对所有会话可见，会话结束后元数据继续存在。会话与会话之间的用户数据、索引和统计信息相互隔离，每个会话只能看到和更改自己提交的数据。 + +全局临时表有两种模式：一种是基于会话级别的（ON COMMIT PRESERVE ROWS），当会话结束时自动清空用户数据；一种是基于事务级别的（ON COMMIT DELETE ROWS）, 当执行commit或rollback时自动清空用户数据。建表时如果没有指定ON COMMIT选项，则缺省为会话级别。与本地临时表不同，全局临时表建表时可以指定非pg_temp_开头的schema。 + +## 特性增强 + +在本地临时表的基础上增加了全局临时表的处理。 + +## 特性约束 + +- 不支持并行扫描 +- 不支持temp tablespace +- 不支持partition +- 不支持GIST索引 +- 不支持User-defined统计信息pg_statistic_ext +- 不支持ON COMMIT DROP +- 不支持hash bucket 聚簇存储 +- 不支持列存 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/12-pseudocolumn-rownum.md b/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/12-pseudocolumn-rownum.md new file mode 100644 index 0000000000000000000000000000000000000000..793cfb21eea32687eec626d08ed19de942399785 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/12-pseudocolumn-rownum.md @@ -0,0 +1,43 @@ +--- +title: 伪列ROWNUM +summary: 伪列ROWNUM +author: Guo Huan +date: 2022-05-07 +--- + +# 伪列ROWNUM + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +ROWNUM为查询出来的每一行记录生成一个序号，从1开始依次递增且不会重复。 + +## 客户价值 + +- 兼容Oracle特性，方便数据库迁移。 +- 与LIMIT特性类似，能够筛选出结果集的前n条记录。 + +## 特性描述 + +ROWNUM（伪列），用于对SQL查询中符合条件的记录进行顺序标注。查询结果第一行ROWNUM为1，第二行为2，依次类推，第n行为n。通常用来筛选出查询结果集中的前n行数据，与MogDB中LIMIT功能类似。 + +## 特性增强 + +在内部执行时，优化器会将ROWNUM重写成LIMIT去执行，加快执行速率。 + +## 特性约束 + +- ROWNUM是伪列，不可作为别名，以免SQL语句出现歧义； +- 创建索引时不可使用ROWNUM。例如：create index index_name on table(rownum); +- 创建表时default值不可为ROWNUM。例如：create table table_name(id int default rownum); +- Where子句中不可使用rownum的别名。例如：select rownum rn from table where rn < 5; +- 在插入数据时不可使用ROWNUM。例如：insert into table values(rownum,’blue’); +- 不可在无表查询中使用ROWNUM。例如：select * from (values(rownum,1)), x(a,b); +- 若 having 子句中含有ROWNUM（且不在聚合函数中）时，group by子句中必须含有ROWNUM（且不在聚合函数中）。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/13-stored-procedure-debugging.md b/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/13-stored-procedure-debugging.md new file mode 100644 index 0000000000000000000000000000000000000000..e3c2b78a62323f7a751d09d5c729e347bcf8dcfa --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/13-stored-procedure-debugging.md @@ -0,0 +1,38 @@ +--- +title: 支持存储过程调试 +summary: 支持存储过程调试 +author: Guo Huan +date: 2022-05-07 +--- + +# 支持存储过程调试 + +## 可获得性 + +本特性自MogDB 1.1.0 版本开始引入。 + +## 特性简介 + +提供调试存储过程的一组API，可以进行断点调试、变量打印等对存储过程的调试功能。 + +## 客户价值 + +提升客户基于MogDB开发存储过程的开发体验。 + +## 特性描述 + +存储过程是数据库中的一种重要对象，主要功能将用户特定功能的SQL语句集进行封装，并方便调用。一个存储过程中往往包含着众多的sql以及过程化的执行结构，根据所需业务的规模大小而定。然而书写一个大的存储过程，往往会伴随着逻辑BUG，仅通过执行存储过程观察结果很难甚至无法发现其中存在的问题，因此需要一个调试工具。 + +存储过程调试工具，提供一组调试接口，可以使存储过程单步执行，在执行过程中设置断点、打印变量等，便于SQL开发者能够及时便捷的发现错误、修正错误，更高效、更快捷、高质量的进行功能开发。 + +## 特性增强 + +无。 + +## 特性约束 + +无。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/14-jdbc-client-load-balancing-and-readwrite-isolation.md b/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/14-jdbc-client-load-balancing-and-readwrite-isolation.md new file mode 100644 index 0000000000000000000000000000000000000000..8681123c66dab6449260ab336646590758d04977 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/14-jdbc-client-load-balancing-and-readwrite-isolation.md @@ -0,0 +1,36 @@ +--- +title: JDBC客户端负载均衡与读写分离 +summary: JDBC客户端负载均衡与读写分离 +author: Guo Huan +date: 2022-05-07 +--- + +# JDBC客户端负载均衡与读写分离 + +## 可获得性 + +本特性自MogDB 2.1.0 版本开始引入。 + +## 特性简介 + +JDBC 客户端提供负载均衡与读写分离能力。 + +## 客户价值 + +可从JDBC客户端测配置负载均衡与读写分离。 + +## 特性描述 + +客户端多节点IP+Port配置,适应多AZ间高可用切换、适应异地容灾切换。支持连接级读写分离配置。支持优先连接只读节点。多个只读节点连接分布均衡。 + +## 特性增强 + +无。 + +## 特性约束 + +无。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/15-in-place-update-storage-engine.md b/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/15-in-place-update-storage-engine.md new file mode 100644 index 0000000000000000000000000000000000000000..dc24b37f56d10b693f6b897060483fcdb3e0e34b --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/15-in-place-update-storage-engine.md @@ -0,0 +1,36 @@ +--- +title: In-place Update存储引擎 +summary: In-place Update存储引擎 +author: Guo Huan +date: 2022-05-07 +--- + +# In-place Update存储引擎 + +## 可获得性 + +本特性自MogDB 2.1.0 版本开始引入。 + +## 特性简介 + +In-place Update存储引擎（原地更新），是MogDB内核新增的一种存储模式。MogDB内核此前的版本使用的行存储引擎是Append Update（追加更新）模式。追加更新对于业务中的增、删以及HOT(HeapOnly Tuple) Update(即同一页面内更新)有很好的表现，但对于跨数据页面的非HOT UPDATE场景，垃圾回收不够高效，Ustore存储引擎可很好解决上述问题。 + +## 客户价值 + +In-place Update存储引擎可有效的降低多次更新元组后占用存储空间问题。 + +## 特性描述 + +新增的In-place update存储引擎很好的解决了Append update存储引擎空间膨胀，元组较大的劣势，高效回滚段的设计是In-place update存储引擎的基础。 + +## 特性增强 + +无。 + +## 特性约束 + +无。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/2-sql-hints.md b/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/2-sql-hints.md new file mode 100644 index 0000000000000000000000000000000000000000..717f98ffc722d5ea2d9e46753722f8def6c2dd52 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/2-sql-hints.md @@ -0,0 +1,40 @@ +--- +title: 支持SQL hint +summary: 支持SQL hint +author: Guo Huan +date: 2022-05-07 +--- + +# 支持SQL hint + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +支持SQL hint影响执行计划生成。 + +## 客户价值 + +提升SQL查询性能。 + +## 特性描述 + +Plan Hint为用户提供了直接影响执行计划生成的手段，用户可以通过指定join顺序，join、stream、scan方法，指定结果行数，指定重分布过程中的倾斜信息等多个手段来进行执行计划的调优，以提升查询的性能。 + +## 特性增强 + +支持planhint设置session级优化器参数。 + +支持指定子查询不展开。 + +支持单query禁用gpc。 + +## 特性约束 + +无。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/3-full-text-indexing.md b/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/3-full-text-indexing.md new file mode 100644 index 0000000000000000000000000000000000000000..e0d8a52b87dccea35dc015c0fed10f7677db97d2 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/3-full-text-indexing.md @@ -0,0 +1,53 @@ +--- +title: 全文索引 +summary: 全文索引 +author: Guo Huan +date: 2022-05-07 +--- + +# 全文索引 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +MogDB中提供的全文索引功能可以对文档进行预处理，并且可以使后续的搜索更快速。 + +## 客户价值 + +MogDB全文索引功能提供了查询可读性文档的能力，并且通过查询相关度将结果进行排序。 + +## 特性描述 + +构建全文索引的预处理过程包括： + +- 将文档解析成token。 + + 为每个文档标记不同类别的token是非常有必要的，例如：数字、文字、复合词、电子邮件地址，这样就可以做不同的处理。原则上token的类别依赖于具体的应用，但对于大多数的应用来说，可以使用一组预定义的token类。 + +- 将token转换为词素。 + + 词素像token一样是一个字符串，但它已经标准化处理，这样同一个词的不同形式是一样的。例如，标准化通常包括：将大写字母折成小写字母、删除后缀（如英语中的s或者es）。这将允许通过搜索找到同一个词的不同形式，不需要繁琐地输入所有可能的变形样式。同时，这一步通常会删除停用词。这些停用词通常因为太常见而对搜索无用。（总之，token是文档文本的原片段，而词素被认为是有用的索引和搜索词。）MogDB使用词典执行这一步，且提供了各种标准的词典。 + +- 保存搜索优化后的预处理文档。 + + 比如，每个文档可以呈现为标准化词素的有序组合。伴随词素，通常还需要存储词素位置信息以用于邻近排序。因此文档包含的查询词越密集其排序越高。词典能够对token如何标准化做到细粒度控制。使用合适的词典，可以定义不被索引的停用词。 + +## 特性增强 + +无。 + +## 特性约束 + +MogDB的全文检索功能当前限制约束是： + +- 每个分词长度必须小于2KB。 +- tsvector结构（分词+位置）的长度必须小于1MB。 +- tsvector的位置值必须大于0，且小于等于16,383。 +- 每个分词在文档中位置数必须小于256，若超过将舍弃后面的位置信息。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/4-copy-interface-for-error-tolerance.md b/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/4-copy-interface-for-error-tolerance.md new file mode 100644 index 0000000000000000000000000000000000000000..2ad59d972119ba4663fe957d60472a06bd0f4ce5 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/4-copy-interface-for-error-tolerance.md @@ -0,0 +1,36 @@ +--- +title: Copy接口支持容错机制 +summary: Copy接口支持容错机制 +author: Guo Huan +date: 2022-05-07 +--- + +# Copy接口支持容错机制 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +支持将Copy过程中的部分错误导入到指定的错误表中，并且保持Copy过程不被中断。 + +## 客户价值 + +提升Copy功能的可用性和易用性，提升对于源数据格式异常等常见错误的容忍性和鲁棒性。 + +## 特性描述 + +MogDB提供用于创建函数的封装好的Copy错误表，并允许用户在使用Copy From指令时指定容错选项，使得Copy From语句在执行过程中部分解析、数据格式、字符集等相关的报错不会报错中断事务、而是被记录至错误表中，使得在Copy From的目标文件即使有少量数据错误也可以完成入库操作。用户随后可以在错误表中对相关的错误进行定位以及进一步排查。 + +## 特性增强 + +无。 + +## 特性约束 + +支持容错的具体错误种类请参见《管理指南》中“使用COPY FROM STDIN导入数据 > [处理错误表](3-running-the-COPY-FROM-STDIN-statement-to-import-data#处理错误表)”章节。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/5-partitioning.md b/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/5-partitioning.md new file mode 100644 index 0000000000000000000000000000000000000000..8723a87f169f46e4c54326152c0a35e37aade06f --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/5-partitioning.md @@ -0,0 +1,58 @@ +--- +title: 分区 +summary: 分区 +author: Guo Huan +date: 2022-05-07 +--- + +# 分区 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +数据分区是在一个节点内部对数据按照用户指定的策略做进一步的水平分表，将表按照指定范围划分为多个数据互不重叠的部分。 + +## 客户价值 + +对于大多数用户使用场景，分区表和普通表相比具有以下优点： + +- 改善查询性能：对分区对象的查询可以仅搜索自己关心的分区，提高检索效率。 +- 增强可用性：如果分区表的某个分区出现故障，表在其他分区的数据仍然可用。 +- 均衡I/O：可以把不同的分区映射到不同的磁盘以平衡I/O，改善整个系统性能。 + +## 特性描述 + +目前MogDB数据库支持范围分区表、列表分区表、哈希分区表： + +- 范围分区表：将数据基于范围映射到每一个分区，这个范围是由创建分区表时指定的分区键决定的。这种分区方式是最为常用的。 + + 范围分区功能，即根据表的一列或者多列，将要插入表的记录分为若干个范围（这些范围在不同的分区里没有重叠），然后为每个范围创建一个分区，用来存储相应的数据。 + +- 列表分区表：将数据基于各个分区内包含的键值映射到每一个分区，分区包含的键值在创建分区时指定。 + + 列表分区功能，即根据表的一列，将要插入表的记录中出现的键值分为若干个列表（这些列表在不同的分区里没有重叠），然后为每个列表创建一个分区，用来存储相应的数据。 + +- 哈希分区表：将数据通过哈希映射到每一个分区，每一个分区中存储了具有相同哈希值的记录。 + + 哈希分区功能，即根据表的一列，通过内部哈希算法将要插入表的记录划分到对应的分区中。 + +用户在CREATE TABLE时增加PARTITION参数，即表示针对此表应用数据分区功能。用户可以在实际使用中根据需要调整建表时的分区键，使每次查询结果尽可能存储在相同或者最少的分区内（称为“分区剪枝”），通过获取连续I/O大幅度提升查询性能。 + +实际业务中，时间经常被作为查询对象的过滤条件。因此，用户可考虑选择时间列为分区键，键值范围可根据总数据量、一次查询数据量调整。 + +## 特性增强 + +支持范围分区表的合并功能。 + +List/Hash分区表支持数据交换。 + +## 特性约束 + +无。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/6-support-for-advanced-analysis-functions.md b/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/6-support-for-advanced-analysis-functions.md new file mode 100644 index 0000000000000000000000000000000000000000..a849553f32f25f131c9c699c2ee210c61fc52408 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/6-support-for-advanced-analysis-functions.md @@ -0,0 +1,57 @@ +--- +title: 高级分析函数支持 +summary: 高级分析函数支持 +author: Guo Huan +date: 2022-05-07 +--- + +# 高级分析函数支持 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +无。 + +## 客户价值 + +我们提供窗口函数来进行数据高级分析处理。窗口函数将一个表中的数据进行预先分组，每一行属于一个特定的组，然后在这个组上进行一系列的关联分析计算。这样可以挖掘出每一个元组在这个集合里的一些属性和与其他元组的关联信息。 + +## 特性描述 + +简单举例说明窗口分析功能：分析某一部门内每个人的薪水和部门平均薪水的对比。 + +``` +SELECT depname, empno, salary, avg(salary) OVER (PARTITION BY depname) FROM empsalary; +depname | empno | salary | avg +-----------+-------+--------+----------------------- +develop | 11 | 5200 | 5020.0000000000000000 +develop | 7 | 4200 | 5020.0000000000000000 +develop | 9 | 4500 | 5020.0000000000000000 +develop | 8 | 6000 | 5020.0000000000000000 +develop | 10 | 5200 | 5020.0000000000000000 +personnel | 5 | 3500 | 3700.0000000000000000 +personnel | 2 | 3900 | 3700.0000000000000000 +sales | 3 | 4800 | 4866.6666666666666667 +sales | 1 | 5000 | 4866.6666666666666667 +sales | 4 | 4800 | 4866.6666666666666667 +(10 rows) +``` + +可以看到，通过这个avg(salary) OVER (PARTITION BY depname)分析函数，每一个人的薪水和与部门的平均薪水很容易计算出来。 + +目前，系统支持row_number()，rank()，dense_rank()，percent_rank()，cume_dist()，ntile()，lag()，lead()，first_value()，last_value()，nth_value()分析函数。具体的函数用法和语句请参见《参考指南》中“系统函数 > [窗口函数](18-window-functions)”章节。 + +## 特性增强 + +无。 + +## 特性约束 + +无。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/7-materialized-view.md b/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/7-materialized-view.md new file mode 100644 index 0000000000000000000000000000000000000000..0bf10b93589652af52d3e41b669002251bdf5221 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/7-materialized-view.md @@ -0,0 +1,36 @@ +--- +title: 物化视图 +summary: 物化视图 +author: Guo Huan +date: 2022-05-07 +--- + +# 物化视图 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +物化视图实际上就是一种特殊的物理表，物化视图是相对普通视图而言的。普通视图是虚拟表，应用的局限性较大，任何对视图的查询实际上都是转换为对SQL语句的查询，性能并没有实际上提高。而物化视图实际上就是存储SQL所执行语句的结果，起到缓存的效果。 + +## 客户价值 + +使用物化视图功能提升查询效率。 + +## 特性描述 + +支持全量物化视图和增量物化视图，全量物化视图只支持全量更新。增量物化视图可实现异步更新功能，用户可通过执行语句把新增数据刷新到物化视图中。 + +## 特性增强 + +无。 + +## 特性约束 + +仅支持基表简单过滤查询和UNION ALL语句。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/8-hyperloglog.md b/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/8-hyperloglog.md new file mode 100644 index 0000000000000000000000000000000000000000..93eebd2ea64b7629d8bcfc14306f52fc8e94195c --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/8-hyperloglog.md @@ -0,0 +1,38 @@ +--- +title: 支持HyperLogLog +summary: 支持HyperLogLog +author: Guo Huan +date: 2022-05-07 +--- + +# 支持HyperLogLog + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +通过使用HyperLogLog相关函数，计算唯一值个数Count（Distinct），提升性能。 + +## 客户价值 + +提升AP/TP类查询的性能。 + +## 特性描述 + +HLL（HyperLogLog）是统计数据集中唯一值个数的高效近似算法。它有着计算速度快，节省空间的特点，不需要直接存储集合本身，而是存储一种名为HLL的数据结构。每当有新数据加入进行统计时，只需要把数据经过哈希计算并插入到HLL中，最后根据HLL就可以得到结果。 + +HLL在计算速度和所占存储空间上都占优势。在时间复杂度上，Sort算法需要排序至少O(nlogn)的时间，虽说Hash算法和HLL一样扫描一次全表O(n)的时间就可以得出结果，但是存储空间上，Sort算法和Hash算法都需要先把原始数据存起来再进行统计，会导致存储空间消耗巨大。而对HLL来说不需要存原始数据，只需要维护HLL数据结构，所以占用空间始终是1280字节常数级别。 + +## 特性增强 + +无。 + +## 特性约束 + +无。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/9-creating-an-index-online.md b/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/9-creating-an-index-online.md new file mode 100644 index 0000000000000000000000000000000000000000..7a0b28001d8029bf5aa90fbc86fcf5547fc4ad3f --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/enterprise-level-features/9-creating-an-index-online.md @@ -0,0 +1,40 @@ +--- +title: 在线添加索引 +summary: 在线添加索引 +author: Guo Huan +date: 2022-05-07 +--- + +# 在线添加索引 + +## 可获得性 + +本特性自MogDB 1.1.0 版本开始引入。 + +## 特性简介 + +通过create index concurrently语法，以不阻塞DML的方式在线创建索引。 + +## 客户价值 + +创建索引时指定concurrently关键字，可以实现创建过程中不阻塞DML，不阻塞用户在线业务。 + +## 特性描述 + +创建索引时，一般会阻塞其他语句对该索引所依赖表的访问。指定concurrently关键字，对表加的是ShareUpdateExclusiveLock锁，可以实现创建过程中不阻塞DML。 + +在线添加索引，需要执行先后两次对该表的权标扫描来完成build，第一次扫描的时候创建索引，不阻塞读写操作；第二次扫描的时候合并更新第一次扫描到目前为止发生的变更。由于需要执行两次对表的扫描和build，而且必须等待现有的所有可能对该表执行修改的事务结束。这意味着该索引的创建比正常耗时更长，同时因此带来的CPU和I/O消耗对其他业务也会造成影响。 + +## 特性增强 + +无。 + +## 特性约束 + +- 在线添加索引时只能指定一个索引的名称。 +- 普通CREATE INDEX命令可以在事务内执行，但是CREATE INDEX CONCURRENTLY不可以在事务内执行。 +- 列存表、分区表和临时表不支持CONCURRENTLY方式创建索引。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/high-availability/1-primary-standby.md b/product/zh/docs-mogdb/v2.1/characteristic-description/high-availability/1-primary-standby.md new file mode 100644 index 0000000000000000000000000000000000000000..202f27e3f90ab1a7a062b73d7e8c78843b443515 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/high-availability/1-primary-standby.md @@ -0,0 +1,42 @@ +--- +title: 主备机 +summary: 主备机 +author: Guo Huan +date: 2022-05-07 +--- + +# 主备机 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始支持DN主备。 + +## 特性简介 + +为了保证故障的可恢复，需要将数据写多份，设置主备多个副本，通过日志进行数据同步，可以实现节点故障、停止后重启等情况下，MogDB能够保证故障之前的数据无丢失，满足ACID特性。 + +## 客户价值 + +主备机功能可以支持主机故障时切换到备机，数据不丢失，业务可以快速恢复。 + +## 特性描述 + +主备环境可以支持主备从和一主多备两种模式。主备从模式下，备机需要重做日志，可以升主，而从备只能接收日志，不可以升主。而在一主多备模式下，所有的备机都需要重做日志，都可以升主。主备从主要用于大数据分析类型的OLAP系统，能够节省一定的存储资源。而一主多备提供更高的容灾能力，更加适合于大批量事务处理的OLTP系统。 + +当主备机均正常可用时，主备之间可以通过switchover进行角色切换；当主机故障后，可以通过failover对备机进行升主。 + +初始化安装或者备份恢复等场景中，需要根据主机重建备机的数据，此时需要build功能，将主机的数据和WAL日志发送到备机。主机故障后重新以备机的角色加入时，也需要build功能将其数据和日志与新主拉齐。另外，在在线扩容的场景中，需要通过build来同步元数据到新节点上的实例。build包含全量build和增量build，全量build要全部依赖主机数据进行重建，拷贝的数据量比较大，耗时比较长，而增量build只拷贝差异文件，拷贝的数据量比较小，耗时比较短。一般情况下，优先选择增量build来进行故障恢复，如果增量build失败，再继续执行全量build，直至故障恢复。 + +为了实现所有实例的高可用容灾能力，除了以上对DN设置主备多个副本，MogDB还提供了其他一些主备容灾能力，使得实例故障后可以尽快地恢复，不中断业务，将因为硬件、软件和人为造成的故障对业务的影响降到最低，以保证业务的连续性。 + +## 特性增强 + +无。 + +## 特性约束 + +无。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/high-availability/10-adding-or-deleting-a-standby-server.md b/product/zh/docs-mogdb/v2.1/characteristic-description/high-availability/10-adding-or-deleting-a-standby-server.md new file mode 100644 index 0000000000000000000000000000000000000000..0954f4d14b6485488b8503c8593bee75cf047c30 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/high-availability/10-adding-or-deleting-a-standby-server.md @@ -0,0 +1,59 @@ +--- +title: 备机增加删除 +summary: 备机增加删除 +author: Guo Huan +date: 2022-05-07 +--- + +# 备机增加删除 + +## 可获得性 + +本特性自MogDB 2.0.0 版本开始引入。 + +## 特性简介 + +支持新增和删除备节点。 + +## 客户价值 + +- 当主数据库的读取压力较大，或者用户想要提升数据库灾难恢复能力，需要新增备节点。 +- 当集群中的某些备节点发生故障无法短时间内进行修复时，为保证集群状态正常，用户可以先将故障节点删除。 + +## 特性描述 + +- 支持从单机或者一主多备状态下最多扩容至一主八备，支持新增级联备机，支持在集群中存在故障备机的情况下新增备节点。 +- 支持从一主多备删除至单节点，支持备机故障的情况下将其删除。 +- 支持在线新增和删除备节点，即可以在不影响主机业务的情况下进行新增和删除备节点。 + +## 特性增强 + +无。 + +## 特性约束 + +新增备机的约束： + +- 数据库主机上存在MogDB镜像包。 +- 在新增的扩容备机上创建好与主机上相同的用户和用户组。 +- 已存在的数据库节点和新增的扩容节点之间需要建立好root用户互信以及数据库管理用户的互信。 +- 正确配置xml文件，在已安装数据库配置文件的基础上，添加需要扩容的备机信息。 +- 只能使用root用户执行扩容命令。 +- 不允许同时在主节点上执行gs_dropnode命令删除其他备机。 +- 执行扩容命令前需要导入主机数据库的环境变量。 +- 扩容备机的操作系统与主机保持一致。 +- 操作过程中不允许同时在其他备节点上执行主备倒换或者故障倒换的操作。 + +删除备机的约束： + +- 删除备节点的操作只能在主节点上执行。 +- 操作过程中不允许同时在其他备节点上执行主备倒换或者故障倒换的操作。 +- 不允许同时在主节点上执行gs_expansion命令进行扩容。 +- 不允许同时执行2次相同的gs_dropnode命令。 +- 执行删除操作前，需要确保主节点和备节点之间建立好数据库管理用户的互信。 +- 需要使用数据库管理用户执行该命令。 +- 执行命令前需要通过source命令导入主机数据库的环境变量。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/high-availability/2-logical-replication.md b/product/zh/docs-mogdb/v2.1/characteristic-description/high-availability/2-logical-replication.md new file mode 100644 index 0000000000000000000000000000000000000000..53d2aba5033547f389c97b1bc9d38f6504861965 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/high-availability/2-logical-replication.md @@ -0,0 +1,37 @@ +--- +title: 逻辑复制 +summary: 逻辑复制 +author: Guo Huan +date: 2022-05-07 +--- + +# 逻辑复制 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +MogDB提供逻辑解码功能，将物理日志反解析为逻辑日志。通过DRS等逻辑复制工具将逻辑日志转化为SQL语句，到对端数据库回放，达到异构数据库同步数据的功能。目前支持MogDB数据库与MySQL数据库、Oracle数据库之间的单向、双向逻辑复制。 + +## 客户价值 + +逻辑复制可以为数据库数据实时迁移、双库双活、支持滚动升级提供解决方案。 + +## 特性描述 + +DN通过物理日志反解析为逻辑日志，DRS等逻辑复制工具从DN抽取逻辑日志转换为SQL语句，到对端数据库（MySQL）回放。逻辑复制工具同时从MySQL数据库抽取逻辑日志，反解析为SQL语句之后回放到MogDB，达到异构数据库同步数据的目的。 + +## 特性增强 + +- MogDB 1.1.0逻辑解码新增全量+增量抽取日志的方案。 +- MogDB 1.1.0逻辑解码新增备机支持逻辑解码。 + +## 特性约束 + +不支持列存复制，不支持DDL复制。 + +## 依赖关系 + +依赖于逻辑复制工具对逻辑日志进行解码。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/high-availability/3-online-node-replacement.md b/product/zh/docs-mogdb/v2.1/characteristic-description/high-availability/3-online-node-replacement.md new file mode 100644 index 0000000000000000000000000000000000000000..929dfbf80a832a16a1ac4b45603c73e93e65dc3e --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/high-availability/3-online-node-replacement.md @@ -0,0 +1,36 @@ +--- +title: 在线节点替换 +summary: 在线节点替换 +author: Guo Huan +date: 2022-05-07 +--- + +# 在线节点替换 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +数据库内某节点出现硬件故障造成节点不可用或者实例状态不正常，当数据库没有加锁，通过节点替换或修复故障实例来恢复数据库的过程中，支持用户DML操作，有限场景支持用户DDL操作。 + +## 客户价值 + +随着企业数据规模不断增大，节点数量急剧增加，硬件损坏概率相应增加，物理节点替换修复成为日常运维工作的常态。传统的离线节点替换方式无法满足客户业务不中断需求，日常运维操作中，经常的业务中断将给客户带来重大损失。而目前业界数据库产品在节点替换的过程中，或者需要中断业务，或者只允许部分操作，均不能满足大规模数据情况下，常态物理节点替换的需求。 + +## 特性描述 + +数据库内某节点出现硬件故障造成节点不可用或者实例状态不正常，通过节点替换或修复故障实例来恢复数据库的过程中，支持用户DML操作，有限场景支持用户DDL操作。 + +## 特性增强 + +无。 + +## 特性约束 + +目前节点替换已支持用户业务在线DDL：在节点替换窗口期内，支持用户DML操作，有限场景支持用户DDL操作。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/high-availability/4-logical-backup.md b/product/zh/docs-mogdb/v2.1/characteristic-description/high-availability/4-logical-backup.md new file mode 100644 index 0000000000000000000000000000000000000000..c978f096b45aa7b44d6b0171ce293e7085abc7ff --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/high-availability/4-logical-backup.md @@ -0,0 +1,40 @@ +--- +title: 逻辑备份 +summary: 逻辑备份 +author: Guo Huan +date: 2022-05-07 +--- + +# 逻辑备份 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +支持将数据库内用户表的数据以通用格式备份到指定的存储介质中。 + +## 客户价值 + +通过逻辑备份特性，可以达成以下目的： + +- 将用户数据备份到可靠性更高的存储介质中，提升用户表数据的可靠性。 +- 通过采用通用的数据格式，支持跨版本恢复和异构恢复。 +- 可以用于冷数据的归档。 + +## 特性描述 + +MogDB提供逻辑备份能力，可以将用户表的数据以通用的text或者csv格式备份到本地磁盘文件中，并在同构/异构数据库中恢复该用户表的数据。 + +## 特性增强 + +无。 + +## 特性约束 + +逻辑备份的约束条件请参见《参考指南》中“[gs_dump](5-gs_dump)”章节。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/high-availability/5-physical-backup.md b/product/zh/docs-mogdb/v2.1/characteristic-description/high-availability/5-physical-backup.md new file mode 100644 index 0000000000000000000000000000000000000000..62ffc1500b99e6467c51e0b52287280e8d8a7b57 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/high-availability/5-physical-backup.md @@ -0,0 +1,52 @@ +--- +title: 物理备份 +summary: 物理备份 +author: Guo Huan +date: 2022-05-07 +--- + +# 物理备份 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +支持将整个数据库的数据以内部格式备份到指定的存储介质中。 + +## 客户价值 + +通过物理备份特性，可以达成以下目的： + +- 整个数据库的数据备份到可靠性更高的存储介质中，提升系统整体的可靠性。 +- 通过采用数据库内部的数据格式，极大提升备份恢复性能。 +- 可以用于冷数据的归档。 + +典型的物理备份策略和应用场景如下： + +- 周一，执行数据库全量备份 +- 周二，以周一全量备份为基准点，执行增量备份 +- 周三，以周二增量备份为基准点，执行增量备份 +- … +- 周日，以周六增量备份为基准点，执行增量备份 + +上述备份策略以一个星期为周期。 + +## 特性描述 + +MogDB 1.1.0 提供物理备份能力，可以将整个数据库的数据以数据库内部格式备份到本地磁盘文件、OBS对象、NBU对象或爱数对象中，并在同构数据库中恢复整个数据库的数据。在基础之上，还提供压缩、流控、断点续备等高阶功能。 + +物理备份主要分为全量备份和增量备份，区别如下：全量备份包含备份时刻点上数据库的全量数据，耗时时间长（和数据库数据总量成正比），自身即可恢复出完整的数据库；增量备份只包含从指定时刻点之后的增量修改数据，耗时时间短（和增量数据成正比，和数据总量无关），但是必须要和全量备份数据一起才能恢复出完整的数据库。 + +## 特性增强 + +支持全量备份和增量备份同时执行。 + +## 特性约束 + +物理备份的约束条件请参见《管理指南》中“备份与恢复 > [物理备份恢复](1-2-br)”章节相关约束。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/high-availability/6-automatic-job-retry-upon-failure.md b/product/zh/docs-mogdb/v2.1/characteristic-description/high-availability/6-automatic-job-retry-upon-failure.md new file mode 100644 index 0000000000000000000000000000000000000000..6a8a8292c02de1a9476ad7770a1757ae48556078 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/high-availability/6-automatic-job-retry-upon-failure.md @@ -0,0 +1,93 @@ +--- +title: 作业失败自动重试 +summary: 作业失败自动重试 +author: Guo Huan +date: 2022-05-07 +--- + +# 作业失败自动重试 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +批处理作业中，在由于网络异常、锁冲突等出错情况下能够保证作业自动重试。 + +## 客户价值 + +网络异常、锁冲突等出错场景下无需人工干预，自动对出错查询进行重试，提高了可用性。 + +## 特性描述 + +MogDB数据库提供作业重试机制gsql Retry。 + +gsql Retry，对于需要进行重试的错误使用唯一的错误码（SQL STATE）进行标识。对客户端工具gsql进行功能增强，使用错误码配置文件（retry_errcodes.conf，置于和gsql同级的安装目录下）对需要重试的错误列表进行配置。gsql提供如下元命令来开启/关闭错误重试功能，重试次数设为范围为5~10，缺省值为5。 + +``` +set RETRY [number] +``` + +打开该功能时gsql会读取上述配置文件，错误重试控制器通过容器记录错误码列表。如果打开后出现配置文件中某一个错误，控制器会将每次缓存的查询语句重新发送给服务端进行重试，直到成功或超过重试次数报错。 + +## 特性增强 + +无。 + +## 特性约束 + +- 功能范围限制： + + 仅能提高故障发生时SQL语句执行成功率，不能保证100%的执行成功。 + +- 错误类型约束： + + SQL语句出错时能够被识别和重试的错误，仅限在表1中定义的错误。 + + **表 1** 支持的错误类型列表 + + | 错误类型 | 错误码 | 备注 | + | :----------------------------------------------------------- | :----- | :----------------------------------------------------------- | + | 对端连接重置（CONNECTION_RESET_BY_PEER） | YY001 | TCP通信错误：Connection reset by peer | + | 对端流重置（STREAM_CONNECTION_RESET_BY_PEER） | YY002 | TCP通信错误：Stream connection reset by peer（DN和DN间通信） | + | 锁等待超时（LOCK_WAIT_TIMEOUT） | YY003 | 锁超时，Lock wait timeout | + | 连接超时（CONNECTION_TIMED_OUT） | YY004 | TCP通信错误，Connection timed out | + | 查询设置错误（SET_QUERY_ERROR） | YY005 | SET命令发送失败，Set query error | + | 超出逻辑内存（OUT_OF_LOGICAL_MEMORY） | YY006 | 内存申请失败，Out of logical memory | + | 通信库内存分配（SCTP_MEMORY_ALLOC） | YY007 | SCTP通信错误，Memory allocate error | + | 无通信库缓存数据（SCTP_NO_DATA_IN_BUFFER） | YY008 | SCTP通信错误，SCTP no data in buffer | + | 通信库释放内存关闭（SCTP_RELEASE_MEMORY_CLOSE） | YY009 | SCTP通信错误，Release memory close | + | SCTP、TCP断开（SCTP_TCP_DISCONNECT） | YY010 | SCTP、TCP通信错误，SCTP、TCP disconnect | + | 通信库断开（SCTP_DISCONNECT） | YY011 | SCTP通信错误，SCTP disconnect | + | 通信库远程关闭（SCTP_REMOTE_CLOSE） | YY012 | SCTP通信错误，Stream closed by remote | + | 等待未知通信库通信（SCTP_WAIT_POLL_UNKNOW） | YY013 | 等待未知通信库通信，SCTP wait poll unknow | + | 无效快照（SNAPSHOT_INVALID） | YY014 | 快照非法，Snapshot invalid | + | 通讯接收信息错误（ERRCODE_CONNECTION_RECEIVE_WRONG） | YY015 | 连接获取错误，Connection receive wrong | + | 内存耗尽（OUT_OF_MEMORY） | 53200 | 内存耗尽，Out of memory | + | 连接异常（CONNECTION_EXCEPTION） | 08000 | 连接出现错误，和DN的通讯失败，Connection exception | + | 管理员关闭系统（ADMIN_SHUTDOWN） | 57P01 | 管理员关闭系统，Admin shutdown | + | 关闭远程流接口（STREAM_REMOTE_CLOSE_SOCKET） | XX003 | 关闭远程套接字，Stream remote close socket | + | 重复查询编号（ERRCODE_STREAM_DUPLICATE_QUERY_ID） | XX009 | 重复查询，Duplicate query id | + | stream查询并发更新同一行（ERRCODE_STREAM_CONCURRENT_UPDATE） | YY016 | stream查询并发更新同一行，Stream concurrent update | + +- 语句类型约束： + + 支持单语句存储过程、函数、匿名块。不支持事务块中的语句。 + +- 存储过程语句约束： + + - 包含EXCEPTION的存储过程，如果在执行过程中（包含语句块执行和EXCEPTION中的语句执行）错误被抛出，可以retry，如果报错被EXCEPTION捕获则不能retry。 + - 不支持使用全局变量的高级包。 + - 不支持DBE_TASK。 + - 不支持PKG_UTIL文件操作。 + +- 数据导入约束： + + - 不支持COPY FROM STDIN语句。 + - 不支持gsql \copy from元命令。 + - 不支持JDBC CopyManager copyIn导入数据。 + +## 依赖关系 + +该特性依赖gsql工具端可以正常工作、错误列表配置正确。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/high-availability/7-ultimate-rto.md b/product/zh/docs-mogdb/v2.1/characteristic-description/high-availability/7-ultimate-rto.md new file mode 100644 index 0000000000000000000000000000000000000000..6a608eb28cdd9c08f42fbbaac11a974686d5af36 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/high-availability/7-ultimate-rto.md @@ -0,0 +1,39 @@ +--- +title: 极致RTO +summary: 极致RTO +author: Guo Huan +date: 2022-05-07 +--- + +# 极致RTO + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +- 支撑数据库主机重启后快速恢复的场景。 +- 支撑主机与同步备机通过日志同步，加速备机回放的场景。 + +## 客户价值 + +当业务压力过大时，备机的回放速度跟不上主机的速度。在系统长时间的运行后，备机上会出现日志累积。当主机故障后，数据恢复需要很长时间，数据库不可用，严重影响系统可用性。 + +开启极致RTO（Recovery Time Object，恢复时间目标），减少了主机故障后数据的恢复时间，提高了可用性。 + +## 特性描述 + +极致RTO开关开启后，xlog日志回放建立多级流水线，提高并发度，提升日志回放速度。 + +## 特性增强 + +无。 + +## 特性约束 + +极致RTO只关注同步备机的RTO是否满足需求。极致RTO本身有流控效果，无需打开流控功能开关。本特性不支持备机读。如果查询备机可能导致备机宕机。本特性不支持主备从场景。如果配置成支持主备从（配置了replication_type=0），数据库无法拉起。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/high-availability/8-cascaded-standby-server.md b/product/zh/docs-mogdb/v2.1/characteristic-description/high-availability/8-cascaded-standby-server.md new file mode 100644 index 0000000000000000000000000000000000000000..e3cce4a690383cccdcff97e4b10fe7462331d208 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/high-availability/8-cascaded-standby-server.md @@ -0,0 +1,45 @@ +--- +title: 级联备机 +summary: 级联备机 +author: Guo Huan +date: 2022-05-07 +--- + +# 级联备机 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +本特性主要基于当前一主多备的架构，在此基础上，支持级联备机连接备机。 + +## 客户价值 + +一主多备架构在特性业务场景下，无法支持足够灵活的结构。多机房部署，不足以满足HA切换下的完整结构（主次机房三个实例，再次机房两到三个实例）。若备机个数变多，对主机可能导致压力较大。针对业务实时性要求较低的查询，可落到级联备中。因此，我们需要提供级联备能力。 + +## 特性描述 + +主机通过发送日志给备机实现同步，备机通过发送日志给级联备实现同步，其中主备之间日志同步可配置为同步或异步，备机与级联备之间只能以异步方式。 + +目前一主多备方式架构，主机通过walsender线程向备机同步日志，备机通过walreceiver线程接受日志，并刷到本地盘，备机读取redo日志，完成主备之间的数据同步。主备机之间walsender与walreceiver一一对应。备机与级联备通过walsender与walreceiver线程异步方式同步日志，降低主机流复制的压力。 + +## 特性增强 + +无。 + +## 特性约束 + +- 级联备只能从备机同步数据，不能直接从主机同步。 +- 级联备不支持直接从备机build数据，目前只能从主机build数据。如果备机需要全量build，会导致级联备也要重新全量build。 +- 级联备为异步同步。 +- 级联备不支持promote。 +- 级联备不支持notify。 +- 目前不提供查询主 - 备 - 级联备集群总体架构，需要根据主找到备，再通过备找到级联备。 +- 级联备不能有再次级联备。 +- 极致RTO开启时，不支持级联备。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/high-availability/9-delayed-replay.md b/product/zh/docs-mogdb/v2.1/characteristic-description/high-availability/9-delayed-replay.md new file mode 100644 index 0000000000000000000000000000000000000000..dab3131094e636a76e9a03a1fb8f320e35cc0229 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/high-availability/9-delayed-replay.md @@ -0,0 +1,46 @@ +--- +title: 延时回放 +summary: 延时回放 +author: Guo Huan +date: 2022-05-07 +--- + +# 延时回放 + +## 可获得性 + +本特性自MogDB 2.0.0 版本开始引入。 + +## 特性简介 + +支持备机延时回放。 + +## 客户价值 + +默认情况下，备机会尽快恢复来自于主机的XLOG记录。这个功能允许备机将回放延时一段指定的时间后进行回放。提供一份可查询一段时间之前的数据副本，可以更方便的纠正用户误操作等错误。 + +## 特性描述 + +支持通过配置GUC参数recovery_min_apply_delay设定延时时间，使某个备机延时一段时间回放来自主机的事务日志。 + +参数取值范围：整型，0~INT_MAX，单位为毫秒。 + +参数默认值：0（不增加延迟） + +## 特性增强 + +无。 + +## 特性约束 + +- recovery_min_apply_delay参数主节点设置无效，必须设置在需要延迟的备节点上。 +- 延迟时间是根据主服务器上事务提交的时间戳与备机上的当前时间来计算，因此需要保证主备系统时钟一致。 +- 没有事务的操作不会被延迟。 +- 主备切换之后，原主机若需延迟，需要再手动配置此参数。 +- 当synchronous_commit被设置为remote_apply时，同步复制会受到这个延时的影响，每一个COMMIT都需要等待备机回放结束后才会返回。 +- 使用这个特性也会让hot_standby_feedback被延迟，这可能导致主服务器的膨胀，两者一起使用时要小心。 +- 主机执行了持有AccessExclusive锁的DDL操作，比如DROP和TRUNCATE操作，在备机延迟回放该条记录期间，在备机上对该操作对象执行查询操作会等待锁释放之后才会返回。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/1-cbo-optimizer.md b/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/1-cbo-optimizer.md new file mode 100644 index 0000000000000000000000000000000000000000..0c11ecfbcbe01f9e6b0ae19faa07fcbfe7f9fdd5 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/1-cbo-optimizer.md @@ -0,0 +1,36 @@ +--- +title: CBO优化器 +summary: CBO优化器 +author: Guo Huan +date: 2022-05-07 +--- + +# CBO优化器 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +MogDB优化器是基于代价的优化（Cost-Based Optimization，简称CBO）。 + +## 客户价值 + +MogDB CBO优化器能够在众多计划中依据代价选出最高效的执行计划，最大限度的满足客户业务要求。 + +## 特性描述 + +在CBO优化器模型下，数据库根据表的元组数、字段宽度、NULL记录比率、DISTINCT值、MCV值、HB值等表的特征值，以及一定的代价计算模型，计算出每一个执行步骤的不同执行方式的输出元组数和执行代价（cost），进而选出整体执行代价最小、首元组返回代价最小的执行方式进行执行。 + +## 特性增强 + +无。 + +## 特性约束 + +无。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/10-xlog-no-lock-flush.md b/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/10-xlog-no-lock-flush.md new file mode 100644 index 0000000000000000000000000000000000000000..5fea498c508682efafeb5012648ffc3a20cd86ec --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/10-xlog-no-lock-flush.md @@ -0,0 +1,36 @@ +--- +title: Xlog no Lock Flush +summary: Xlog no Lock Flush +author: Guo Huan +date: 2022-05-07 +--- + +# Xlog no Lock Flush + +## 可获得性 + +本特性自MogDB 1.1.0 版本开始引入。 + +## 特性简介 + +取消WalInsertLock争抢及WalWriter专用磁盘写入线程。 + +## 客户价值 + +在保持原有XLog功能不变的基础上，进一步提升系统性能。 + +## 特性描述 + +对WalInsertLock进行优化，利用LSN（Log Sequence Number）及LRC（Log Record Count）记录了每个backend的拷贝进度，取消WalInsertLock机制。在backend将日志拷贝至WalBuffer时，不用对WalInsertLock进行争抢，可直接进行日志拷贝操作。并利用专用的WalWriter写日志线程，不需要backend线程自身来保证xlog的Flush。 + +## 特性增强 + +无。 + +## 特性约束 + +无。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/11-parallel-page-based-redo-for-ustore.md b/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/11-parallel-page-based-redo-for-ustore.md new file mode 100644 index 0000000000000000000000000000000000000000..266305bc08408c6057e376647bc8c49fb2c51cc8 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/11-parallel-page-based-redo-for-ustore.md @@ -0,0 +1,36 @@ +--- +title: Parallel Page-based Redo For Ustore +summary: Parallel Page-based Redo For Ustore +author: Guo Huan +date: 2022-05-07 +--- + +# Parallel Page-based Redo For Ustore + +## 可获得性 + +本特性自MogDB 1.1.0 版本开始引入。 + +## 特性简介 + +优化Ustore Inplace Update WAL log写入，Ustore DML Operation回放提高并行度。 + +## 客户价值 + +对于Update的WAL使用空间减少，Ustore DML Operation回放提高并行度。 + +## 特性描述 + +通过利用Prefix和suffix来减少update WAL log的写入，通过把回放线程分多个类型来解决Ustore DML WAL大多都是多页面回放问题；同时把Ustore的数据页面回放按照blkno去分发。 + +## 特性增强 + +无。 + +## 特性约束 + +无。 + +## 依赖关系 + +依赖于Ustore引擎。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/2-llvm.md b/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/2-llvm.md new file mode 100644 index 0000000000000000000000000000000000000000..1b58d6b9e2745050a0a617ba1cbbc64dd451a4f8 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/2-llvm.md @@ -0,0 +1,36 @@ +--- +title: 支持LLVM +summary: 支持LLVM +author: Guo Huan +date: 2022-05-07 +--- + +# 支持LLVM + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +MogDB的LLVM（Low Level Virtual Machine）技术，提供了查询动态编译执行的能力。 + +## 客户价值 + +通过对查询动态编译执行，大幅提高重查询性能。 + +## 特性描述 + +MogDB借助LLVM提供的库函数，依据查询执行计划树，将原本在执行器阶段才会确定查询实际执行路径的过程提前到执行初始化阶段，从而规避原本查询执行时候伴随的函数调用、逻辑条件分支判断以及大量的数据读取等问题，以达到提升查询性能的目的。 + +## 特性增强 + +无。 + +## 特性约束 + +无。 + +## 依赖关系 + +依赖于LLVM开源组件，当前使用的开源组件版本为10.0.0。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/3-vectorized-engine.md b/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/3-vectorized-engine.md new file mode 100644 index 0000000000000000000000000000000000000000..6d0a7b526925b6ec10e027fe2140ad349f20f410 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/3-vectorized-engine.md @@ -0,0 +1,43 @@ +--- +title: 向量化引擎 +summary: 向量化引擎 +author: Guo Huan +date: 2022-05-07 +--- + +# 向量化引擎 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +MogDB提供向量化引擎，通常用在OLAP数据仓库类系统。主要是因为分析型系统通常是数据处理密集型，基本上都是采用顺序方式来访问表中大部分的数据，然后再进行计算，最后将计算结果输出给用户。 + +## 客户价值 + +通过批量计算，大幅提高复杂类查询性能。 + +## 特性描述 + +传统的数据库查询执行都是采用一次一数组（tuple）的pipeline执行模式，因此CPU的大部分处理时间不是用来处理数据，而是遍历查询操作树。这种情况下CPU的有效利用率不高，同时也会导致低指令缓存性能和频繁跳转。更加糟糕的是，这种方式的执行，不能够利用现代硬件的新优化特征来加速查询的执行。在执行引擎中，另外一个解决方案就是改变一次一数组（tuple）为一次一列的模式。这也是我们向量化执行引擎的一个基础。 + +向量化引擎是跟列存储技术绑定的，因为列存储时每列数据存储在一起，可以认为这些数据是以数组的方式存储的。基于这样的特征，当该列数据需要进行某一同样操作，可以通过一个循环来高效完成对这个数据块各个值的计算。 + +向量化执行引擎的优势在于： + +- 可以减少节点间的调度，提高CPU的利用率。 +- 因为相同类型的数据放在一起，可以更容易的利用硬件与编译的新优化特征。 + +## 特性增强 + +无。 + +## 特性约束 + +无。 + +## 依赖关系 + +依赖于列存储。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/4-hybrid-row-column-store.md b/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/4-hybrid-row-column-store.md new file mode 100644 index 0000000000000000000000000000000000000000..c9cf1377521f956a0985063b191c39420d040218 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/4-hybrid-row-column-store.md @@ -0,0 +1,72 @@ +--- +title: 行列混合存储 +summary: 行列混合存储 +author: Guo Huan +date: 2022-05-07 +--- + +# 行列混合存储 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +MogDB支持行存储和列存储两种存储模型，用户可以根据具体的使用场景，建表时选择行存储还是列存储表。 + +一般情况下，如果表的字段比较多（即大宽表），查询中涉及到列不很多的情况下，适合列存储。列存储方式如图1所示。如果表的字段个数比较少，查询大部分字段，那么选择行存储比较好。 + +**图 1** 列存储示意图 + +![img](https://cdn-mogdb.enmotech.com/docs-media/mogdb/characteristic-description/hybrid-row-column-store.png) + +## 客户价值 + +在大宽表、数据量比较大的场景中，查询经常关注某些列，行存储引擎查询性能比较差。例如，气象局的场景，单表有200~800个列，查询经常访问10个列，在类似这样的场景下，向量化执行技术和列存储引擎可以极大的提升性能和减少存储空间。 + +## 特性描述 + +表有行存表和列存表两种存储模型。两种存储模型各有优劣，建议根据实际情况选择。 + +- 行存表 + + 默认创建表的类型。数据按行进行存储，即一行数据紧挨着存储。行存表支持完整的增删改查。适用于对数据需要经常更新的场景。 + +- 列存表 + +数据按列进行存储，即一列所有数据紧挨着存储。单列查询IO小，比行存表占用更少的存储空间。适合数据批量插入、更新较少和以查询为主统计分析类的场景。列存表不适合点查询，insert插入单条记录性能差。 + +行存表和列存表的选择依据如下： + +- 更新频繁程度 + + 数据如果频繁更新，选择行存表。 + +- 插入频繁程度 + + 频繁的少量插入，选择行存表。一次插入大批量数据，选择列存表。 + +- 表的列数 + + 表的列数很多，选择列存表。 + +- 查询的列数 + + 如果每次查询时，只涉及了表的少数（小于总列数的50%）几个列，选择列存表。 + +- 压缩率 + + 列存表比行存表压缩率高。但高压缩率会消耗更多的CPU资源。 + +## 特性增强 + +无。 + +## 特性约束 + +无。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/5-adaptive-compression.md b/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/5-adaptive-compression.md new file mode 100644 index 0000000000000000000000000000000000000000..e1d7f41a7b1cc5e80d80d29e869b696daddd0c0e --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/5-adaptive-compression.md @@ -0,0 +1,42 @@ +--- +title: 自适应压缩 +summary: 自适应压缩 +author: Guo Huan +date: 2022-05-07 +--- + +# 自适应压缩 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +数据压缩是当前数据库采用的主要技术。数据类型不同，适用于它的压缩算法不同。对于相同类型的数据，其数据特征不同，采用不同的压缩算法达到的效果也不相同。自适应压缩正是从数据类型和数据特征出发，采用相应的压缩算法，实现了良好的压缩比、快速的入库性能以及良好的查询性能。 + +## 客户价值 + +主要应用场景为数据入库和频繁的海量数据查询。 在数据入库场景中，自适应压缩可以大幅度地减少数据量，成倍提高IO操作效率，并将数据分类压缩以获得更快的入库性能。当用户进行数据查询时，少量的IO操作和快速的数据解压可以加快数据获取的速率，从而在更短的时间内得到查询结果。 + +## 特性描述 + +目前，数据库已实现了列存储上RLE、DELTA、BYTEPACK/BITPACK、LZ4、ZLIB、LOCAL DICTIONARY等多种压缩算法。数据库支持的数据类型与压缩算法的映射关系如下表所示。 + +| - | RLE | DELTA | BITPACK/BYTEPACK | LZ4 | ZLIB | LOCAL DICTIONARY | +| :----------------------------------------------------------- | :--- | :---- | :--------------- | :--- | :--- | :--------------- | +| smallint/int/bigint/oiddecimal/real/doublemoney/time/date/timestamp | √ | √ | √ | √ | √ | - | +| tinterval/interval/time with time zone/ | - | - | - | - | √ | - | +| numeric/char/varchar/text/nvarchar2
以及其他支持数据类型 | √ | √ | √ | √ | √ | √ | + +## 特性增强 + +支持对压缩算法进行不同压缩水平的调整。 + +## 特性约束 + +无。 + +## 依赖关系 + +开源压缩软件LZ4/ZLIB。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/6-sql-by-pass.md b/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/6-sql-by-pass.md new file mode 100644 index 0000000000000000000000000000000000000000..c76a03ad578ea0d3400eea49c64050896c738922 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/6-sql-by-pass.md @@ -0,0 +1,36 @@ +--- +title: SQL by pass +summary: SQL by pass +author: Guo Huan +date: 2022-05-07 +--- + +# SQL by pass + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +通过对OLTP场景典型查询的定制化执行方案来提高查询性能。 + +## 客户价值 + +提升OLTP类查询的性能。 + +## 特性描述 + +在典型的OLTP场景中，简单查询占了很大一部分比例。这种查询的特征是只涉及单表和简单表达式的查询，因此为了加速这类查询，提出了SQL by pass框架，在parse层对这类查询做简单的模式判别后，进入到特殊的执行路径里，跳过经典的执行器执行框架，包括算子的初始化与执行、表达式与投影等经典框架，直接重写一套简洁的执行路径，并且直接调用存储接口，这样可以大大加速简单查询的执行速度。 + +## 特性增强 + +无。 + +## 特性约束 + +无。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/7-kunpeng-numa-architecture-optimization.md b/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/7-kunpeng-numa-architecture-optimization.md new file mode 100644 index 0000000000000000000000000000000000000000..4f2e6936b4e6fe345299baae8bfb241c197a167f --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/7-kunpeng-numa-architecture-optimization.md @@ -0,0 +1,40 @@ +--- +title: 鲲鹏NUMA架构优化 +summary: 鲲鹏NUMA架构优化 +author: Guo Huan +date: 2022-05-07 +--- + +# 鲲鹏NUMA架构优化 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +鲲鹏NUMA架构优化，主要面向鲲鹏处理器架构特点、ARMv8指令集等，进行相应的系统优化，涉及到操作系统、软件架构、锁并发、日志、原子操作、Cache访问等一系列的多层次优化，从而大幅提升了MogDB数据库在鲲鹏平台上的处理性能。 + +## 客户价值 + +数据库的处理性能，例如每分钟处理交易量（Transaction Per Minute），是数据库竞争力的关键性能指标，在同等硬件成本的条件下，数据库能提供的处理性能越高，就可以提供给用户更多的业务处理能力，从而降低客户的使用成本。 + +## 特性描述 + +- MogDB根据鲲鹏处理机的多核NUMA架构特点，进行一系列针对性NUMA架构相关优化。一方面尽量减少跨核内存访问的时延问题，另一方面充分发挥鲲鹏多核算力优势，所提供的关键技术包括重做日志批插、热点数据NUMA分布、CLog分区等，大幅提升TP系统的处理性能。 +- MogDB基于鲲鹏芯片所使用的ARMv8.1架构，利用LSE扩展指令集实现高效的原子操作，有效提升CPU利用率，从而提升多线程间同步性能、XLog写入性能等。 +- MogDB基于鲲鹏芯片提供的更宽的L3缓存cacheline，针对热点数据访问进行优化，有效提高缓存访问命中率，降低Cache缓存一致性维护开销，大幅提升系统整体的数据访问性能。 +- 鲲鹏920，2P服务器（64cores*2，内存768 GB），网络10 GE，IO为4块NVME PCIE SSD时，TPCC为1000warehouse，性能是150万tpmC。 + +## 特性增强 + +- 支持重做日志批插，分区CLog，提升鲲鹏平台下的数据库处理性能。 +- 支持LSE扩展指令集的原子操作，提升多线程同步性能。 + +## 特性约束 + +无。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/8-high-concurrency-of-thread-pools.md b/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/8-high-concurrency-of-thread-pools.md new file mode 100644 index 0000000000000000000000000000000000000000..53d9b1577d2937652d1083b76f93a0f5ffe88e5a --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/8-high-concurrency-of-thread-pools.md @@ -0,0 +1,37 @@ +--- +title: 支持线程池高并发 +summary: 支持线程池高并发 +author: Guo Huan +date: 2022-05-07 +--- + +# 支持线程池高并发 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +通过线程池化技术来支撑数据库大并发稳定运行。 + +## 客户价值 + +支撑客户大并发下，系统整体吞吐平稳。 + +## 特性描述 + +线程池技术的整体设计思想是线程资源池化、并且在不同连接之间复用。系统在启动之后会根据当前核数或者用户配置启动固定一批数量的工作线程，一个工作线程会服务一到多个连接会话，这样把会话和线程进行了解耦。因为工作线程数是固定的，因此在高并发下不会导致线程的频繁切换，而由数据库层来进行线程的调度管理。 + +## 特性增强 + +- MogDB 1.1.0版本引入该特性。 +- MogDB 1.1.0版本实现了线程池的动态扩缩容。 + +## 特性约束 + +无。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/9-smp-for-parallel-execution.md b/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/9-smp-for-parallel-execution.md new file mode 100644 index 0000000000000000000000000000000000000000..5ec4a934f147682232f10ca9cf7374590681f9e4 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/high-performance/9-smp-for-parallel-execution.md @@ -0,0 +1,44 @@ +--- +title: SMP并行执行 +summary: SMP并行执行 +author: Guo Huan +date: 2022-05-07 +--- + +# SMP并行执行 + +## 可获得性 + +本特性自MogDB 1.1.0版本开始引入。 + +## 特性简介 + +MogDB的SMP并行技术是一种利用计算机多核CPU架构来实现多线程并行计算，以充分利用CPU资源来提高查询性能的技术。 + +## 客户价值 + +SMP并行技术充分利用了系统多核的能力，来提高重查询的性能。 + +## 特性描述 + +在复杂查询场景中，单个查询的执行较长，系统并发度低，通过SMP并行执行技术实现算子级的并行，能够有效减少查询执行时间，提升查询性能及资源利用率。SMP并行技术的整体实现思想是对于能够并行的查询算子，将数据分片，启动若干个工作线程分别计算，最后将结果汇总，返回前端。SMP并行执行增加数据交互算子**Stream**，实现多个工作线程之间的数据交互，确保查询的正确性，完成整体的查询。 + +## 特性增强 + +无。 + +## 特性约束 + +- 索引扫描不支持并行执行。 +- MergeJoin不支持并行执行。 +- WindowAgg order by不支持并行执行。 +- cursor不支持并行执行。 +- 存储过程和函数内的查询不支持并行执行。 +- 不支持子查询subplan和initplan的并行，以及包含子查询的算子的并行。 +- 查询语句中带有median操作的查询不支持并行执行。 +- 带全局临时表的查询不支持并行执行。 +- 物化视图的更新不支持并行执行。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/maintainability/1-gray-upgrade.md b/product/zh/docs-mogdb/v2.1/characteristic-description/maintainability/1-gray-upgrade.md new file mode 100644 index 0000000000000000000000000000000000000000..9402531ce563d127f43a8a5f1c6e5476d557b505 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/maintainability/1-gray-upgrade.md @@ -0,0 +1,36 @@ +--- +title: 灰度升级 +summary: 灰度升级 +author: Guo Huan +date: 2022-05-07 +--- + +# 灰度升级 + +## 可获得性 + +本特性自MogDB 2.0.0版本开始引入。 + +## 特性简介 + +灰度升级支持全业务操作，一次性升级所有节点。 + +## 客户价值 + +灰度升级提供了一种在线升级的方式，保证在不中断业务的情况下，对所有的节点进行升级。 + +## 特性描述 + +灰度升级是一种支持所有节点升级的在线升级方式。目前灰度升级只涉及数据库二进制的替换，为了尽可能降低对于业务的影响，采用同一节点两套二进制同时存在的方式，使用软连接切换的方式来进行进程版本的切换升级（闪断一次，10秒以内）。 + +## 特性增强 + +无。 + +## 特性约束 + +无。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/maintainability/2-workload-diagnosis-report.md b/product/zh/docs-mogdb/v2.1/characteristic-description/maintainability/2-workload-diagnosis-report.md new file mode 100644 index 0000000000000000000000000000000000000000..37ed7ae684e0ccd328206b75ae562c450c26bed6 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/maintainability/2-workload-diagnosis-report.md @@ -0,0 +1,72 @@ +--- +title: 支持WDR诊断报告 +summary: 支持WDR诊断报告 +author: Guo Huan +date: 2022-05-07 +--- + +# 支持WDR诊断报告 + +## 可获得性 + +本特性自MogDB 1.1.0 版本开始引入。 + +## 特性简介 + +WDR报告提供数据库性能诊断报告，该报告基于基线性能数据和增量数据两个版本，从性能变化得到性能报告。 + +## 客户价值 + +- WDR报表是长期性能问题最主要的诊断手段。基于SNAPSHOT的性能基线，从多维度做性能分析，能帮助DBA掌握系统负载繁忙程度、各个组件的性能表现及性能瓶颈。 +- SNAPSHOT也是后续性能问题自诊断和自优化建议的重要数据来源。 + +## 特性描述 + +WDR（Workload Diagnosis Report）基于两次不同时间点系统的性能快照数据，生成这两个时间点之间的性能表现报表，用于诊断数据库内核的性能故障。 + +使用generate_wdr_report(…) 可以生成基于两个性能快照的性能报告。 + +WDR主要依赖两个组件： + +- SNAPSHOT性能快照：性能快照可以配置成按一定时间间隔从内核采集一定量的性能数据，持久化在用户表空间。任何一个SNAPSHOT可以作为一个性能基线，其他SNAPSHOT与之比较的结果，可以分析出与基线的性能表现。 +- WDR Reporter：报表生成工具基于两个SNAPSHOT，分析系统总体性能表现，并能计算出更多项具体的性能指标在这两个时间段之间的变化量，生成SUMMARY 和DETAIL两个不同级别的性能数据。如表1、表2所示。 + +**表 1** SUMMARY级别诊断报告 + +| 诊断类别 | 描述 | +| :------------------------------ | :----------------------------------------------------------- | +| Database Stat | 主要用于评估当前数据库上的负载和IO状况，负载和IO状况是衡量TP系统最重要的特性。包含当前连接到该数据库的session，提交、回滚的事务数，读取的磁盘块的数量，高速缓存中已经发现的磁盘块的次数，通过数据库查询返回、抓取、插入、更新、删除的行数，冲突、死锁发生的次数，临时文件的使用量，IO读写时间等。 | +| Load Profile | 从时间，IO，事务，SQL几个维度评估当前系统负载的表现。包含作业运行elapse time、CPU time，事务日质量，逻辑和物理读的量，读写IO次数、大小，登入登出次数，SQL、事务执行量，SQL P80、P95响应时间等。 | +| Instance Efficiency Percentages | 用于评估当前系统的缓存的效率。主要包含数据库缓存命中率。 | +| Events | 用于评估当前系统内核关键资源，关键事件的性能。主要包含数据库内核关键事件的发生次数，事件的等待时间。 | +| Wait Classes | 用于评估当前系统关键事件类型的性能。主要包含数据内核在主要的等待事件的种类上的发布：STATUS、LWLOCK_EVENT、LOCK_EVENT、IO_EVENT。 | +| CPU | 主要包含CPU在用户态、内核态、Wait IO、空闲状态下的时间发布。 | +| IO Profile | 主要包含数据库Database IO次数、Database IO数据量、Redo IO次数、Redo IO量。 | +| Memory Statistics | 包含最大进程内存、进程已经使用内存、最大共享内存、已经使用共享内存大小等。 | + +**表 2** DETAIL级别诊断报告 + +| 诊断类别 | 描述 | +| :--------------------- | :----------------------------------------------------------- | +| Time Model | 主要用于评估当前系统在时间维度的性能表现。包含系统在各个阶段上消耗的时间：内核时间、CPU时间、执行时间、解析时间、编译时间、查询重写时间、计划生成时间、网络时间、IO时间。 | +| SQL Statistics | 主要用于SQL语句性能问题的诊断。包含归一化的SQL的性能指标在多个维度上的排序：Elapsed Time、CPU Time、Rows Returned、Tuples Reads、Executions、Physical Reads、Logical Reads。这些指标的种类包括：执行时间，执行次数、行活动、Cache IO等。 | +| Wait Events | 主要用于系统关键资源，关键时间的详细性能诊断。包含所有关键事件在一段时间内的表现，主要是事件发生的次数，消耗的时间。 | +| Cache IO Stats | 用于诊断用户表和索引的性能。包含所有用户表、索引上的文件读写，缓存命中。 | +| Utility status | 用于诊断后端作业性能的诊断。包含页面操作，复制等后端操作的性能。 | +| Object stats | 用于诊断数据库对象的性能。包含用户表、索引上的表、索引扫描活动，insert、update、delete活动，有效行数量，表维护操作的状态等。 | +| Configuration settings | 用于判断配置是否有变更。包含当前所有配置参数的快照。 | +| SQL detail | 显示unique query text信息。 | + +## 特性增强 + +无。 + +## 特性约束 + +- WDR snapshot性能快照会采集不同database的性能数据，如果数据库实例中有大量的database或者大量表，做一次WDR snapshot会花费很长时间。 +- 如果在大量DDL期间做WDR snapshot可能造成WDR snapshot失败。 +- 在drop database时，做WDR snapshot可能造成WDR snapshot失败。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/maintainability/3-slow-sql-diagnosis.md b/product/zh/docs-mogdb/v2.1/characteristic-description/maintainability/3-slow-sql-diagnosis.md new file mode 100644 index 0000000000000000000000000000000000000000..cdcb1d13354f1cf3217b1b2c1c61c0accd1ddd21 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/maintainability/3-slow-sql-diagnosis.md @@ -0,0 +1,137 @@ +--- +title: 慢SQL诊断 +summary: 慢SQL诊断 +author: Guo Huan +date: 2022-05-07 +--- + +# 慢SQL诊断 + +## 可获得性 + +本特性自MogDB 1.1.0 版本开始引入。 + +重构前慢SQL相关视图已废弃，包括dbe_perf. gs_slow_query_info、dbe_perf.gs_slow_query_history、dbe_perf.global_slow_query_hisotry、dbe_perf.global_slow_query_info。 + +## 特性简介 + +慢SQL诊断提供诊断慢SQL所需要的必要信息，帮助开发者回溯执行时间超过阈值的SQL，诊断SQL性能瓶颈。 + +## 客户价值 + +慢SQL诊断提供给用户对于慢SQL所需的详细信息，用户无需通过复现就能离线诊断特定慢SQL的性能问题。表和函数接口方便用户统计慢SQL指标，对接第三方平台。 + +## 特性描述 + +慢SQL能根据用户提供的执行时间阈值（log_min_duration_statement），记录所有超过阈值的执行完毕的作业信息。 + +慢SQL提供表和函数两种维度的查询接口。用户从接口中能查询到作业的执行计划，开始、结束执行时间，执行查询的语句，行活动，内核时间，CPU时间，执行时间，解析时间，编译时间，查询重写时间，计划生成时间，网络时间，IO时间，网络开销，锁开销等。以上所有信息都是脱敏的。 + +## 特性增强 + +增加对慢SQL指标信息，安全性（脱敏），执行计划，查询接口的增强。 + +``` +执行命令查看数据库实例中SQL语句执行信息 +gsql> select * from dbe_perf.get_global_full_sql_by_timestamp(start_timestamp, end_timestamp); +例如： +MogDB=# select * from DBE_PERF.get_global_full_sql_by_timestamp('2020-12-01 09:25:22', '2020-12-31 23:54:41'); +-[ RECORD 1 ]--------+--------------------------------------------------------------------------------------------------------------- +--------------------------------------------------------------- +node_name | dn_6001_6002_6003 +db_name | postgres +schema_name | "$user",public +origin_node | 1938253334 +user_name | user_dj +application_name | gsql +client_addr | +client_port | -1 +unique_query_id | 3671179229 +debug_query_id | 72339069014839210 +query | select name, setting from pg_settings where name in (?) +start_time | 2020-12-19 16:19:51.216818+08 +finish_time | 2020-12-19 16:19:51.224513+08 +slow_sql_threshold | 1800000000 +transaction_id | 0 +thread_id | 139884662093568 +session_id | 139884662093568 +n_soft_parse | 0 +n_hard_parse | 1 +query_plan | Datanode Name: dn_6001_6002_6003 + | Function Scan on pg_show_all_settings a (cost=0.00..12.50 rows=5 width=64) + | Filter: (name = '***'::text) +... + +执行命令查看数据库实例中慢SQL语句执行信息 +gsql> select * from dbe_perf.get_global_slow_sql_by_timestamp(start_timestamp, end_timestamp); +MogDB=# select * from DBE_PERF.get_global_slow_sql_by_timestamp('2020-12-01 09:25:22', '2020-12-31 23:54:41'); +-[ RECORD 1 ]--------+--------------------------------------------------------------------------------------------------- +node_name | dn_6001_6002_6003 +db_name | postgres +schema_name | "$user",public +origin_node | 1938253334 +user_name | user_dj +application_name | gsql +client_addr | +client_port | -1 +unique_query_id | 2165004317 +debug_query_id | 72339069014839319 +query | select * from DBE_PERF.get_global_slow_sql_by_timestamp(?, ?); +start_time | 2020-12-19 16:23:20.738491+08 +finish_time | 2020-12-19 16:23:20.773714+08 +slow_sql_threshold | 10000 +transaction_id | 0 +thread_id | 139884662093568 +session_id | 139884662093568 +n_soft_parse | 10 +n_hard_parse | 8 +query_plan | Datanode Name: dn_6001_6002_6003 + | Result (cost=1.01..1.02 rows=1 width=0) + | InitPlan 1 (returns $0) + | -> Seq Scan on pgxc_node (cost=0.00..1.01 rows=1 width=64) + | Filter: (nodeis_active AND ((node_type = '***'::"char") OR (node_type = '***'::"char"))) +... + +查看当前节点SQL语句执行信息 +gsql> select * from statement_history; +例如： +MogDB=# select * from statement_history; +-[ RECORD 1 ]--------+--------------------------------------------------------------------------------------------------------------- +--------------------------------------------------------------- +db_name | postgres +schema_name | "$user",public +origin_node | 1938253334 +user_name | user_dj +application_name | gsql +client_addr | +client_port | -1 +unique_query_id | 3671179229 +debug_query_id | 72339069014839210 +query | select name, setting from pg_settings where name in (?) +start_time | 2020-12-19 16:19:51.216818+08 +finish_time | 2020-12-19 16:19:51.224513+08 +slow_sql_threshold | 1800000000 +transaction_id | 0 +thread_id | 139884662093568 +session_id | 139884662093568 +n_soft_parse | 0 +n_hard_parse | 1 +query_plan | Datanode Name: dn_6001_6002_6003 + | Function Scan on pg_show_all_settings a (cost=0.00..12.50 rows=5 width=64) + | Filter: (name = '***'::text) +``` + +## 特性约束 + +- 目前的SQL跟踪信息，基于正常的执行逻辑。执行失败的SQL，其跟踪信息不具有准确的参考价值。 +- 节点重启，可能导致该节点的数据丢失。 +- SQL语句执行完立即退出会话，可能会丢失该会话未刷新到系统表中的数据。 +- 通过GUC参数设置收集SQL语句的数量，如果超过阈值，新的SQL语句执行信息不会被收集。 +- 通过GUC参数设置单条SQL语句收集的锁事件详细信息的最大字节数，如果超过阈值，新的锁事件详细信息不会被收集。 +- 通过异步刷新方式刷新用户执行中的SQL信息，所以用户Query执行结束后，存在查询相关视图函数结果短暂时延。 +- 部分指标信息（行活动、Cache/IO、时间分布等）依赖于dbe_perf.statement视图收集，如果该视图对应记录数超过预定大小（依赖GUC：instr_unique_sql_count），则本特性可能不收集相关指标。 +- statement_history表相关函数以及视图中details字段为二进制格式，如果需要解析详细内容，请使用对应函数pg_catalog.statement_detail_decode(details, 'plaintext', true)。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/maintainability/4-session-performance-diagnosis.md b/product/zh/docs-mogdb/v2.1/characteristic-description/maintainability/4-session-performance-diagnosis.md new file mode 100644 index 0000000000000000000000000000000000000000..bc4d577704bd88daee0a7fc66d301251dc1d0e41 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/maintainability/4-session-performance-diagnosis.md @@ -0,0 +1,106 @@ +--- +title: Session性能诊断 +summary: Session性能诊断 +author: Guo Huan +date: 2022-05-07 +--- + +# Session性能诊断 + +## 可获得性 + +本特性自MogDB 1.1.0 版本开始引入。 + +## 特性简介 + +Session性能诊断提供给用户Session级别的性能问题诊断。 + +## 客户价值 + +- 查看最近用户Session最耗资源的事件。 +- 查看最近比较占资源的SQL把资源都消耗在哪些等待事件上。 +- 查看最近比较耗资源的Session把资源都花费在哪些等待事件上。 +- 查看最近最耗资源的用户的信息。 +- 查看过去Session相互阻塞的等待关系。 + +## 特性描述 + +Session性能诊断提供对当前系统所有活跃Session进行诊断的能力。由于实时采集所有活跃Session的指标对用户负载的影响加大，因此采取Session快照的技术对活跃Session的指标进行采样。 + +从采样中统计出活跃Session的统计指标，这些统计指标从客户端信息、执行开始、结束时间，SQL文本，等待事件，当前数据库对象等维度，反映活跃Session的基本信息，状态，持有的资源。 + +基于概率统计的活跃Session信息，可以帮助用户诊断系统中哪些Session消耗了更多的CPU、内存资源，哪些数据库对象是热对象，哪些SQL消耗了更多的关键事件资源等，从而定位出有问题Session，SQL，数据库设计。 + +Session采样数据分为两级，如图1所示： + +1. 第一级为实时信息，存储在内存中，展示最近几分钟的活跃Session信息，具有最高的精度； +2. 第二级为持久化历史信息，存储在磁盘文件中，展示过去很长一段时间的历史活跃Session信息，从内存数据中抽样而来，适合长时间跨度的统计分析。 + +**图 1** Session性能诊断原理 + + ![Session性能诊断原理](https://cdn-mogdb.enmotech.com/docs-media/mogdb/characteristic-description/session-performance-diagnosis-1.png) + +部分使用场景如下所示： + +1. 查看session之间的阻塞关系 + + ``` + select sessionid, block_sessionid from pg_thread_wait_status; + ``` + +2. 采样blocking session信息 + + ``` + select sessionid, block_sessionid from DBE_PERF.local_active_session; + ``` + +3. Final blocking session展示 + + ``` + select sessionid, block_sessionid, final_block_sessionid from DBE_PERF.local_active_session; + ``` + +4. 最耗资源的wait event + + ``` + SELECT s.type, s.event, t.count + FROM dbe_perf.wait_events s, ( + SELECT event, COUNT (*) + FROM dbe_perf.local_active_session + WHERE sample_time > now() - 5 / (24 * 60) + GROUP BY event)t WHERE s.event = t.event ORDER BY count DESC; + ``` + +5. 查看最近五分钟较耗资源的session把资源都花费在哪些event上 + + ``` + SELECT sessionid, start_time, event, count + FROM ( + SELECT sessionid, start_time, event, COUNT(*) + FROM dbe_perf.local_active_session + WHERE sample_time > now() - 5 / (24 * 60) + GROUP BY sessionid, start_time, event) as t ORDER BY SUM(t.count) OVER \(PARTITION BY t. sessionid, start_time)DESC, t.event; + ``` + +6. 最近五分钟比较占资源的SQL把资源都消耗在哪些event上 + + ``` + SELECT query_id, event, count + FROM ( + SELECT query_id, event, COUNT(*) + FROM dbe_perf.local_active_session + WHERE sample_time > now() - 5 / (24 * 60) + GROUP BY query_id, event) t ORDER BY SUM (t.count) OVER (PARTITION BY t.query_id ) DESC, t.event DESC; + ``` + +## 特性增强 + +无。 + +## 特性约束 + +无。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/characteristic-description/maintainability/5-system-kpi-aided-diagnosis.md b/product/zh/docs-mogdb/v2.1/characteristic-description/maintainability/5-system-kpi-aided-diagnosis.md new file mode 100644 index 0000000000000000000000000000000000000000..a9c33ac848967f784c4423fca379dd7969a3a31c --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/characteristic-description/maintainability/5-system-kpi-aided-diagnosis.md @@ -0,0 +1,66 @@ +--- +title: 系统KPI辅助诊断 +summary: 系统KPI辅助诊断 +author: Guo Huan +date: 2022-05-07 +--- + +# 系统KPI辅助诊断 + +## 可获得性 + +本特性自MogDB 1.1.0 版本开始引入。 + +## 特性简介 + +KPI是内核组件或者整体性能关键指标的视图呈现，基于这些指标，用户可以了解到系统运行的实时或者历史状态。 + +## 客户价值 + +- 系统负载概要诊断 + + 系统负载异常（过载、失速、业务SLA）精准告警，系统负载精准画像。 + +- 系统时间模型概要诊断 + + Instance和Query级别时间模型细分，诊断Instance和Query性能问题根因。 + +- Query性能诊断 + + 数据库级Query概要信息，TopSQL，SQL CPU，IO消耗，执行计划，硬解析过多。 + +- 磁盘IO、索引、buffer性能问题 + +- 连接池，线程池异常 + +- Checkpoint，Redo（RTO）性能问题 + +- 系统I/O、LWLock、Waits性能问题诊断 + + 诊断60+模块，240+关键操作性能问题。 + +- 函数级性能看护诊断（GSTRACE），功能诊断 + + 50+存储和执行层函数trace。 + +## 特性描述 + +MogDB提供涵盖11大类，26个子类的KPI，包括：Instance、File、Object、Workload、Communication、Session、Thread、Cache IO、Lock、Wait Event、Cluster。 + +KPI指标内核的分布如图1所示。 + +**图 1** KPI指标内核分布 + + ![KPI指标内核分布](https://cdn-mogdb.enmotech.com/docs-media/mogdb/characteristic-description/system-kpi-aided-diagnosis.png) + +## 特性增强 + +无。 + +## 特性约束 + +无。 + +## 依赖关系 + +无。 \ No newline at end of file diff --git a/product/zh/docs-mogdb/v2.1/toc.md b/product/zh/docs-mogdb/v2.1/toc.md index 8a3ee2c28ccd0a9f9e721ec482279c67f90b8f6d..0f7cbfef89c0f2ab853367f5259c98ea7ac717da 100644 --- a/product/zh/docs-mogdb/v2.1/toc.md +++ b/product/zh/docs-mogdb/v2.1/toc.md @@ -45,6 +45,78 @@ + [Rust](/quick-start/mogdb-access/use-programming-language-to-access-mogdb/rust.md) + [NodeJS](/quick-start/mogdb-access/use-programming-language-to-access-mogdb/nodejs.md) + [使用样本数据集Mogila](/quick-start/mogila.md) ++ 特性描述 + + 高性能 + + [CBO优化器](/characteristic-description/high-performance/1-cbo-optimizer.md) + + [支持LLVM](/characteristic-description/high-performance/2-llvm.md) + + [向量化引擎](/characteristic-description/high-performance/3-vectorized-engine.md) + + [行列混合存储](/characteristic-description/high-performance/4-hybrid-row-column-store.md) + + [自适应压缩](/characteristic-description/high-performance/5-adaptive-compression.md) + + [SQL by pass](/characteristic-description/high-performance/6-sql-by-pass.md) + + [鲲鹏NUMA架构优化](/characteristic-description/high-performance/7-kunpeng-numa-architecture-optimization.md) + + [支持线程池高并发](/characteristic-description/high-performance/8-high-concurrency-of-thread-pools.md) + + [SMP并行执行](/characteristic-description/high-performance/9-smp-for-parallel-execution.md) + + [Xlog no Lock Flush](/characteristic-description/high-performance/10-xlog-no-lock-flush.md) + + [Parallel Page-based Redo For Ustore](/characteristic-description/high-performance/11-parallel-page-based-redo-for-ustore.md) + + 高可用 + + [主备机](/characteristic-description/high-availability/1-primary-standby.md) + + [逻辑复制](/characteristic-description/high-availability/2-logical-replication.md) + + [在线节点替换](/characteristic-description/high-availability/3-online-node-replacement.md) + + [逻辑备份](/characteristic-description/high-availability/4-logical-backup.md) + + [物理备份](/characteristic-description/high-availability/5-physical-backup.md) + + [作业失败自动重试](/characteristic-description/high-availability/6-automatic-job-retry-upon-failure.md) + + [极致RTO](/characteristic-description/high-availability/7-ultimate-rto.md) + + [级联备机](/characteristic-description/high-availability/8-cascaded-standby-server.md) + + [延时回放](/characteristic-description/high-availability/9-delayed-replay.md) + + [备机增加删除](/characteristic-description/high-availability/10-adding-or-deleting-a-standby-server.md) + + 维护性 + + [灰度升级](/characteristic-description/maintainability/1-gray-upgrade.md) + + [支持WDR诊断报告](/characteristic-description/maintainability/2-workload-diagnosis-report.md) + + [慢SQL诊断](/characteristic-description/maintainability/3-slow-sql-diagnosis.md) + + [Session性能诊断](/characteristic-description/maintainability/4-session-performance-diagnosis.md) + + [系统KPI辅助诊断](/characteristic-description/maintainability/5-system-kpi-aided-diagnosis.md) + + 数据库安全 + + [访问控制模型](/characteristic-description/database-security/1-access-control-model.md) + + [控制权和访问权分离](/characteristic-description/database-security/2-separation-of-control-and-access-permissions.md) + + [数据库认证机制](/characteristic-description/database-security/3-database-encryption-authentication.md) + + [数据加密存储](/characteristic-description/database-security/4-data-encryption-and-storage.md) + + [数据库审计](/characteristic-description/database-security/5-database-audit.md) + + [网络通信安全](/characteristic-description/database-security/6-network-communication-security.md) + + [资源标签机制](/characteristic-description/database-security/7-resource-label.md) + + [统一审计机制](/characteristic-description/database-security/8-unified-audit.md) + + [动态数据脱敏机制](/characteristic-description/database-security/9-dynamic-data-anonymization.md) + + [行级访问控制](/characteristic-description/database-security/10-row-level-access-control.md) + + [用户口令强度校验机制](/characteristic-description/database-security/11-password-strength-verification.md) + + [全密态数据库等值查询](/characteristic-description/database-security/12-equality-query-in-a-fully-encrypted-database.md) + + [账本数据库机制](/characteristic-description/database-security/13-ledger-database-mechanism.md) + + 企业级特性 + + [函数及存储过程支持](/characteristic-description/enterprise-level-features/1-support-for-functions-and-stored-procedures.md) + + [支持SQL hint](/characteristic-description/enterprise-level-features/2-sql-hints.md) + + [全文索引](/characteristic-description/enterprise-level-features/3-full-text-indexing.md) + + [Copy接口支持容错机制](/characteristic-description/enterprise-level-features/4-copy-interface-for-error-tolerance.md) + + [分区](/characteristic-description/enterprise-level-features/5-partitioning.md) + + [高级分析函数支持](/characteristic-description/enterprise-level-features/6-support-for-advanced-analysis-functions.md) + + [物化视图](/characteristic-description/enterprise-level-features/7-materialized-view.md) + + [支持HyperLogLog](/characteristic-description/enterprise-level-features/8-hyperloglog.md) + + [在线添加索引](/characteristic-description/enterprise-level-features/9-creating-an-index-online.md) + + [自治事务](/characteristic-description/enterprise-level-features/10-autonomous-transaction.md) + + [全局临时表](/characteristic-description/enterprise-level-features/11-global-temporary-table.md) + + [伪列ROWNUM](/characteristic-description/enterprise-level-features/12-pseudocolumn-rownum.md) + + [支持存储过程调试](/characteristic-description/enterprise-level-features/13-stored-procedure-debugging.md) + + [JDBC客户端负载均衡与读写分离](/characteristic-description/enterprise-level-features/14-jdbc-client-load-balancing-and-readwrite-isolation.md) + + [In-place Update存储引擎](/characteristic-description/enterprise-level-features/15-in-place-update-storage-engine.md) + + 应用开发接口 + + [支持标准SQL](/characteristic-description/application-development-interfaces/1-standard-sql.md) + + [支持标准开发接口](/characteristic-description/application-development-interfaces/2-standard-development-interfaces.md) + + [PG接口兼容](/characteristic-description/application-development-interfaces/3-postgresql-api-compatibility.md) + + [支持PL/Java](/characteristic-description/application-development-interfaces/4-pl-java.md) + + AI能力 + + [Predictor：AI查询时间预测](/characteristic-description/ai-capabilities/1-predictor-ai-query-time-forecasting.md) + + [X-Tuner：参数调优与诊断](/characteristic-description/ai-capabilities/2-x-tuner-parameter-optimization-and-diagnosis.md) + + [SQLdiag：慢SQL发现](/characteristic-description/ai-capabilities/3-sqldiag-slow-sql-discovery.md) + + [Anomaly-detection：数据库指标采集、预测与异常监控](/characteristic-description/ai-capabilities/4-anomaly-detection-database-indicator-collection-forecasting-and-exception-monitoring.md) + + [Index-advisor：索引推荐](/characteristic-description/ai-capabilities/5-index-advisor-index-recommendation.md) + + [DeepSQL：库内AI算法](/characteristic-description/ai-capabilities/6-deepsql-ai-algorithm-in-the-database.md) + 安装指南 + [容器化安装](/installation-guide/docker-installation/docker-installation.md) + 极简安装 diff --git a/product/zh/docs-mogdb/v2.1/toc_characteristic_description.md b/product/zh/docs-mogdb/v2.1/toc_characteristic_description.md new file mode 100644 index 0000000000000000000000000000000000000000..ea06b347b697c714a7055cebe7cb3a9d3e8fae71 --- /dev/null +++ b/product/zh/docs-mogdb/v2.1/toc_characteristic_description.md @@ -0,0 +1,78 @@ + + + +# MogDB Documentation 2.1 + +## 特性描述 + ++ 高性能 + + [CBO优化器](/characteristic-description/high-performance/1-cbo-optimizer.md) + + [支持LLVM](/characteristic-description/high-performance/2-llvm.md) + + [向量化引擎](/characteristic-description/high-performance/3-vectorized-engine.md) + + [行列混合存储](/characteristic-description/high-performance/4-hybrid-row-column-store.md) + + [自适应压缩](/characteristic-description/high-performance/5-adaptive-compression.md) + + [SQL by pass](/characteristic-description/high-performance/6-sql-by-pass.md) + + [鲲鹏NUMA架构优化](/characteristic-description/high-performance/7-kunpeng-numa-architecture-optimization.md) + + [支持线程池高并发](/characteristic-description/high-performance/8-high-concurrency-of-thread-pools.md) + + [SMP并行执行](/characteristic-description/high-performance/9-smp-for-parallel-execution.md) + + [Xlog no Lock Flush](/characteristic-description/high-performance/10-xlog-no-lock-flush.md) + + [Parallel Page-based Redo For Ustore](/characteristic-description/high-performance/11-parallel-page-based-redo-for-ustore.md) ++ 高可用 + + [主备机](/characteristic-description/high-availability/1-primary-standby.md) + + [逻辑复制](/characteristic-description/high-availability/2-logical-replication.md) + + [在线节点替换](/characteristic-description/high-availability/3-online-node-replacement.md) + + [逻辑备份](/characteristic-description/high-availability/4-logical-backup.md) + + [物理备份](/characteristic-description/high-availability/5-physical-backup.md) + + [作业失败自动重试](/characteristic-description/high-availability/6-automatic-job-retry-upon-failure.md) + + [极致RTO](/characteristic-description/high-availability/7-ultimate-rto.md) + + [级联备机](/characteristic-description/high-availability/8-cascaded-standby-server.md) + + [延时回放](/characteristic-description/high-availability/9-delayed-replay.md) + + [备机增加删除](/characteristic-description/high-availability/10-adding-or-deleting-a-standby-server.md) ++ 维护性 + + [灰度升级](/characteristic-description/maintainability/1-gray-upgrade.md) + + [支持WDR诊断报告](/characteristic-description/maintainability/2-workload-diagnosis-report.md) + + [慢SQL诊断](/characteristic-description/maintainability/3-slow-sql-diagnosis.md) + + [Session性能诊断](/characteristic-description/maintainability/4-session-performance-diagnosis.md) + + [系统KPI辅助诊断](/characteristic-description/maintainability/5-system-kpi-aided-diagnosis.md) ++ 数据库安全 + + [访问控制模型](/characteristic-description/database-security/1-access-control-model.md) + + [控制权和访问权分离](/characteristic-description/database-security/2-separation-of-control-and-access-permissions.md) + + [数据库认证机制](/characteristic-description/database-security/3-database-encryption-authentication.md) + + [数据加密存储](/characteristic-description/database-security/4-data-encryption-and-storage.md) + + [数据库审计](/characteristic-description/database-security/5-database-audit.md) + + [网络通信安全](/characteristic-description/database-security/6-network-communication-security.md) + + [资源标签机制](/characteristic-description/database-security/7-resource-label.md) + + [统一审计机制](/characteristic-description/database-security/8-unified-audit.md) + + [动态数据脱敏机制](/characteristic-description/database-security/9-dynamic-data-anonymization.md) + + [行级访问控制](/characteristic-description/database-security/10-row-level-access-control.md) + + [用户口令强度校验机制](/characteristic-description/database-security/11-password-strength-verification.md) + + [全密态数据库等值查询](/characteristic-description/database-security/12-equality-query-in-a-fully-encrypted-database.md) + + [账本数据库机制](/characteristic-description/database-security/13-ledger-database-mechanism.md) ++ 企业级特性 + + [函数及存储过程支持](/characteristic-description/enterprise-level-features/1-support-for-functions-and-stored-procedures.md) + + [支持SQL hint](/characteristic-description/enterprise-level-features/2-sql-hints.md) + + [全文索引](/characteristic-description/enterprise-level-features/3-full-text-indexing.md) + + [Copy接口支持容错机制](/characteristic-description/enterprise-level-features/4-copy-interface-for-error-tolerance.md) + + [分区](/characteristic-description/enterprise-level-features/5-partitioning.md) + + [高级分析函数支持](/characteristic-description/enterprise-level-features/6-support-for-advanced-analysis-functions.md) + + [物化视图](/characteristic-description/enterprise-level-features/7-materialized-view.md) + + [支持HyperLogLog](/characteristic-description/enterprise-level-features/8-hyperloglog.md) + + [在线添加索引](/characteristic-description/enterprise-level-features/9-creating-an-index-online.md) + + [自治事务](/characteristic-description/enterprise-level-features/10-autonomous-transaction.md) + + [全局临时表](/characteristic-description/enterprise-level-features/11-global-temporary-table.md) + + [伪列ROWNUM](/characteristic-description/enterprise-level-features/12-pseudocolumn-rownum.md) + + [支持存储过程调试](/characteristic-description/enterprise-level-features/13-stored-procedure-debugging.md) + + [JDBC客户端负载均衡与读写分离](/characteristic-description/enterprise-level-features/14-jdbc-client-load-balancing-and-readwrite-isolation.md) + + [In-place Update存储引擎](/characteristic-description/enterprise-level-features/15-in-place-update-storage-engine.md) ++ 应用开发接口 + + [支持标准SQL](/characteristic-description/application-development-interfaces/1-standard-sql.md) + + [支持标准开发接口](/characteristic-description/application-development-interfaces/2-standard-development-interfaces.md) + + [PG接口兼容](/characteristic-description/application-development-interfaces/3-postgresql-api-compatibility.md) + + [支持PL/Java](/characteristic-description/application-development-interfaces/4-pl-java.md) ++ AI能力 + + [Predictor：AI查询时间预测](/characteristic-description/ai-capabilities/1-predictor-ai-query-time-forecasting.md) + + [X-Tuner：参数调优与诊断](/characteristic-description/ai-capabilities/2-x-tuner-parameter-optimization-and-diagnosis.md) + + [SQLdiag：慢SQL发现](/characteristic-description/ai-capabilities/3-sqldiag-slow-sql-discovery.md) + + [Anomaly-detection：数据库指标采集、预测与异常监控](/characteristic-description/ai-capabilities/4-anomaly-detection-database-indicator-collection-forecasting-and-exception-monitoring.md) + + [Index-advisor：索引推荐](/characteristic-description/ai-capabilities/5-index-advisor-index-recommendation.md) + + [DeepSQL：库内AI算法](/characteristic-description/ai-capabilities/6-deepsql-ai-algorithm-in-the-database.md) \ No newline at end of file