From 11d0dc6bc110bb60942852742ac25fa4257d33d6 Mon Sep 17 00:00:00 2001 From: zcp100_zcp100 Date: Fri, 1 Jul 2022 15:54:09 +0800 Subject: [PATCH 1/4] =?UTF-8?q?3.0=E6=96=B0=E7=89=B9=E6=80=A7=E8=8B=B1?= =?UTF-8?q?=E6=96=87?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../about-mogdb/mogdb-new-feature/3.0.0.md | 146 ++++++++++-------- 1 file changed, 82 insertions(+), 64 deletions(-) diff --git a/product/en/docs-mogdb/v3.0/about-mogdb/mogdb-new-feature/3.0.0.md b/product/en/docs-mogdb/v3.0/about-mogdb/mogdb-new-feature/3.0.0.md index a10b37dc..372b109b 100644 --- a/product/en/docs-mogdb/v3.0/about-mogdb/mogdb-new-feature/3.0.0.md +++ b/product/en/docs-mogdb/v3.0/about-mogdb/mogdb-new-feature/3.0.0.md @@ -9,117 +9,127 @@ date: 2022-05-30 ## 1. Version Description -MogDB version 3.0.0 was released on June 30, 2022. MogDB version 3.0.0 is further enhanced based on MogDB version 2.1 and incorporates the new features of openGauss 3.0.0. - -
+MogDB 3.0.0 was released on June 30, 2022. It is an enhanced version based on MogDB 2.1 and incorporated with new features of openGauss 3.0.0. ## 2. New Features -### 2.1 Incorporate new features of openGauss 3.0.0 +### 2.1 Incorporated with New Features of openGauss 3.0.0 - Row-store execution to vectorized execution - Delay of entering the maximum availability mode - Parallel logical decoding -- Cluster Manager (CM) -- global syscache +- CM (Cluster Manager) +- Global SysCache - Publication-Subscription - Foreign key lock enhancement -- Row-store table compression -- Open-source Data Studio -- MySQL to openGauss migration tool Chameleon -- Using ShardingSphere to build a distributed database -- Deploying a distributed database using Kubernetes -- Support ANY permission management +- Support for ANY permission management - DBMind componentized -- Database AI algorithms support XGBoost, multiclass and PCA +- Database AI algorithms supporting XGBoost, multiclass and PCA +- ShardingSphere used to build a distributed database -### 2.2 Cluster Manager （CM） +### 2.2 Asynchronous Commit of Autonomous Transactions -- Provides the ability to monitor the status of primary and standby nodes, network communication failure monitoring, and file system failure monitoring. -- Provides automatic primary and standby switching capability in case of failure. -- Majority voting using the Paxos algorithm to select the master. -- Require at least three servers to have CM components installed. -- The database servers can be one master and one standby. +- Transaction and transaction logging are executed in two threads. This will not make the session worker thread idle in the transaction wait phase. Therefore, the utilization of the session worker thread is improved, and the database performance is further improved on the premise of ensuring data integrity. +- Asynchronous commit of autonomous transaction makes the overall performance of the database improved by 20% to 50% and tpmC increased steadily by 36% in the TPCC compression test. -### 2.3 Performance Enhancement +### 2.3 Parallelism Control Achieved During Index Creation -#### 2.3.1 Transaction asynchronous commit +- MogDB allows you to set the parameter to control parallelism during index creation. It makes index creation efficient and flexible. -- Split transaction execution and transaction log drop into two phases, CPU bound and IO bound, which are executed by different threads to avoid idle CPU resources when executing IO operations, thus improving CPU resource utilization. -- Optimization of transaction asynchronous commits, which can improve transaction throughput by 20% to 50% and TPCC overall performance by 10% to 20%. +### 2.4 Dynamic Partition Pruning -#### 2.3.2 Index Parallel Creation Parallelism Definition +- The dynamic partition pruning function is provided on the basis of static partition pruning. +- Partition pruning is realized in bind variable queries, subqueries, and queries of parameterized nested loops, thereby improving the partition query performance. -- MogDB provides additional parameters to control the parallelism, so you can set the parallelism manually, which is more flexible. +### 2.5 Optimized COPY (Import) -#### 2.3.3 COPY import SIMD acceleration +The SIMD (single instruction, multiple data) vector processing is used combined with the CPU SSE4.2 command set to improve the COPY (import) performance. -- Speeds up the data parsing phase of the COPY command by taking advantage of the CPU's instruction set, thus improving COPY import performance; (x86 CPUs only at this time) +### 2.6 Monitoring the Run Status of SQLs -#### 2.3.4 Dynamic partition reduction +- Execution plans of session-level SQLs are collected and then execution operators are dynamically sampled. +- The query performance problems are efficiently located in slow SQL queries. -- Added support for dynamic partition reductions. In the prepare-execute execution method and in scenarios where partition constraint expressions contain subqueries, partitions are trimmed during the execution phase based on parameters or subquery results to improve partition table query performance. +### 2.7 BRIN Index -#### 2.3.5 Built-in PostGIS supports processing of geographic data +- Compared with BTREE indexes, BRIN (Block Range Index) is very small and scanning the index adds little overhead but the query speed is fast. BRIN is designed for handling very large tables in which certain columns have some natural correlation with their physical location within the table. BRIN index features quick index creation, small overhead, and fast scanning in sequential order. +- This feature is from PostgreSQL. -PostGIS provides spatial information service capabilities such as spatial objects, spatial indexes, spatial manipulation functions, and spatial operators that can be used in: +### 2.8 Bloom Index -- Atmospheric sciences, marine sciences, geology, environmental sciences, traffic management, etc., processing and analyzing complex spatial data and mapping; -- Mobile communications, mobile positioning, mobile Internet; -- Urban management, disaster response, resource development, etc.; +- Bloom index is based on bloom filter. It is applied to scenarios where a table stores a lot of columns and can be queried using any combination of columns. Compared with the traditional index, such as B+ tree, multiple B+ tree indexes need to be created in those scenarios, but one bloom index is enough. +- This feature is from PostgreSQL. -### 2.4 Troubleshooting +### 2.9 PostGIS -#### 2.4.1 Monitor Session level SQL run status +PostGIS is a spatial database system. It provides spatial information service capabilities such as spatial objects, spatial indexes, spatial manipulation functions. It can be used in: -- Collects execution plan trees and dynamically samples execution arithmetic for session-level SQL run states +- Atmospheric sciences, marine sciences, geology, environmental sciences, traffic management, processing and analyzing of complex spatial data, and mapping +- Mobile communications, mobile positioning, and mobile Internet +- Urban management, disaster response, resource development, etc. -#### 2.4.2 Enhanced OM troubleshooting capability +### 2.10 Oracle Compatibility Enhancement -- gstrace enhancements: get more targeted execution paths by adding a **component switch**, which is used to improve debug efficiency. -- gs_check enhancement: the original scene check based on the implementation of the test results to save, as well as the difference between the two test results done at different times to compare. -- gs_watch: When MogDB fails, use this tool to collect OS information, log information and configuration files to locate the problem. -- gs_gucquery: realize MogDB GUC value automatic collection, collation, export and difference comparison. +The whale plugin is incorporated into MogDB and supports the following Oracle functions and packages: -### 2.5 Compatibility Enhancement +**Functions** -#### 2.5.1 Oracle Compatibility Enhancement +- INSTRB +- NLS_CHARSET_ID +- NLS_CHARSET_NAME +- NLS_LOWER +- NLS_UPPER +- ORA_HASH +- REMAINDER +- REPLACE +- SHOW +- SHOW_PARAMETER +- TO_TIMESTAMP +- TO_YMINTERVAL +- TZ_OFFSET +- NULLIF +- Ratio_to_report -- More function support, more built-in package support: dbms_random, dbms_lob, dbms_metadata, etc. -- Support **connect by** syntax -- Reduce the amount of code modifications required to migrate Oracle applications to MogDB. +**Packages** -#### 2.5.2 MySQL Compatibility Enhancement +- dbms_lock +- dbms_job +- dbms_utility +- dbms_random +- dbms_output +- dbms_application_info +- dbms_metadata -- More syntax support: timestamp on update, etc.; more data type compatibility; more function compatibility -- Reduce the amount of code modifications required to migrate MySQL applications to MogDB. +### 2.11 MySQL Compatibility Enhancement -#### 2.5.3 PostgreSQL Compatibility Enhancement +The dolphin plugin is incorporated into MogDB and supports the following syntax and function: -##### 2.5.3.1 Added BRIN INDEX (supported since PostgreSQL 9.5) +- Timestamp on update +- SQL mode (sql_mode_strict and sql_mode_full_group) +- User lock +- Insert function -- Block-wide indexes, compared to precise BTREE indexes, BRIN INDEX provides a balance of a relatively fast query speed with less space consumption -- 1GB table, no index, query single 4s; BTREE index 200MB space, query 4ms; BRIN index 800K, query 58ms. +### 2.12 Enhanced OM Troubleshooting Capability -##### 2.5.3.2 Added BLOOM INDEX (supported since PostgreSQL 9.6) +This troubleshooting capability is enhanced by adding collection of system information, comparing parameters and test results, and adding debugging information, thereby helping users or debugging personnel locate problems efficiently. -- Bloom filtering: true is not necessarily true, false must be false; there is a miscalculation rate, need to recheck (algorithm implementation, not to user recheck) -- For tables with a large number of fields and a combination of query conditions that may also use a large number of fields; only equals queries are supported -- Ordinary indexes for such scenarios require the creation of multiple indexes, which can have a significant impact on space usage and insertion and update speed -- At this point, you can create a BLOOM index uniformly on all these fields that may be used for query, to obtain a balance of space and query speed, 10GB table scan can be completed in about 1s +This feature can be achieved by enhancing the following tools: -
+- gstrace: gets targeted execution paths by adding the component switch function, thereby improving the debugging efficiency. +- gs_check: saves and compares test results on the basis of scenario check. +- gs_watch: When MogDB fails, gs_watch collects OS information, log information, and configuration files to locate the problem. +- gs_gucquery: automatically collects, export, and compare MogDB GUC values. -## 3. Modified Defects +## 3. Fixed Defects -### 3.1 Incorporate openGauss 3.0.0 Modified Defects +### 3.1 Incorporate Fixed Defects in openGauss 3.0.0 - [I4VUXG](https://gitee.com/opengauss/openGauss-server/issues/I4VUXG?from=project-issue): Fixed the data loss issue of unlogged tables. - [I4SF5P](https://gitee.com/opengauss/openGauss-server/issues/I4SF5P?from=project-issue): Fixed the core dump issue occurred by running create extension dblink after the database is compiled and installed in the release version, and the dblink module is compiled and installed. - [I4S74D](https://gitee.com/opengauss/openGauss-server/issues/I4S74D?from=project-issue): Fixed the issue of failing to insert data (5⁄5) into a row-store compressed table using Jmeter when the data volume is greater than 1 GB. The compression type is set to **2**. - [I4N81J](https://gitee.com/opengauss/openGauss-server/issues/I4N81J?from=project-issue): Fixed the issue of failing to synchronize the UPDATE and DELETE operations to subscribers. - [I4YPJQ](https://gitee.com/opengauss/openGauss-server/issues/I4YPJQ?from=project-issue): Fixed the issue of failing to insert varchar constants into MOTs using JDBC. -- [I4PF6G](https://gitee.com/opengauss/openGauss-server/issues/I4PF6G?from=project-issue): Fixed the issue of TPC-C execution failure during foreign key lock enhancement and gray upgrade from 2.0.0. to 2.2.0 (not committed). +- [I4PF6G](https://gitee.com/opengauss/openGauss-server/issues/I4PF6G?from=project-issue): Fixed the issue of TPCC execution failure during foreign key lock enhancement and gray upgrade from 2.0.0. to 2.2.0 (not committed). - [I4WPD1](https://gitee.com/opengauss/openGauss-server/issues/I4WPD1?from=project-issue): Fixed the issue of failing to execute simplified installation because the **openGauss-2.1.0-CentOS-64bit.tar.bz2** file is missing in the decompressed installation package. - [I4L268](https://gitee.com/opengauss/openGauss-server/issues/I4L268?from=project-issue): Fixed the issue of incorrect system catalog **pg_partition** after the partitioned table is truncated for multiple times and then the **vacuum freeze pg_partition** command is executed. - [I3HZJN](https://gitee.com/opengauss/openGauss-server/issues/I3HZJN?from=project-issue): Fixed the issue of incorrect date format when the **copy** command is executed. @@ -131,7 +141,11 @@ PostGIS provides spatial information service capabilities such as spatial object - [I4WRMX](https://gitee.com/opengauss/openGauss-server/issues/I4WRMX?from=project-issue): Fixed the issue of failing to clear data in the **statement_history** table. When the database restarts and the **enable_stmt_track** parameter is disabled, no record should be found in the **statement_history** table. - [I4WOBH](https://gitee.com/opengauss/openGauss-server/issues/I4WOBH?from=project-issue): Fixed the issue of failing to restart the database by setting GUC parameter **pagewriter_sleep** from **360000** to **2000**. -
+### 3.2 Fixed Defects in MogDB 3.0.0 + +- Updating a view may cause a core dump in some scenarios. +- Creating local index tables on second-level partition tables may cause a core dump in some scenarios. +- The CONNECT BY statement may cause a core dump in some scenarios. ## 4. Compatibility @@ -145,3 +159,7 @@ This version supports the following operating system and CPU architecture combin | Kylin V10 | ARM (Kunpeng), X86_64 (Intel, AMD, Hygon, ZHAOXIN) | | UOS V20-D / V20-E | ARM (Kunpeng), X86_64 (Intel, AMD, Hygon, ZHAOXIN) | | UOS V20-A | X86_64 (Intel, AMD, Hygon, ZHAOXIN) | + +## 5. Known Issues + +In the scenario where a partition table uses PBE, dynamic partition pruning may possibly cause core dump. To solve this problem, set **enable_resource_track** to **off**. This parameter has been incorporated into Recommended Parameter Settings. \ No newline at end of file -- Gitee From d7e430b737ada1033685a66908957830f29493bb Mon Sep 17 00:00:00 2001 From: zcp100_zcp100 Date: Fri, 1 Jul 2022 17:28:56 +0800 Subject: [PATCH 2/4] =?UTF-8?q?update(mogdb):mogdb=E6=96=B0=E7=89=B9?= =?UTF-8?q?=E6=80=A7=E4=BF=AE=E6=94=B9?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../en/docs-mogdb/v3.0/about-mogdb/mogdb-new-feature/3.0.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/product/en/docs-mogdb/v3.0/about-mogdb/mogdb-new-feature/3.0.0.md b/product/en/docs-mogdb/v3.0/about-mogdb/mogdb-new-feature/3.0.0.md index 372b109b..805d6c83 100644 --- a/product/en/docs-mogdb/v3.0/about-mogdb/mogdb-new-feature/3.0.0.md +++ b/product/en/docs-mogdb/v3.0/about-mogdb/mogdb-new-feature/3.0.0.md @@ -162,4 +162,4 @@ This version supports the following operating system and CPU architecture combin ## 5. Known Issues -In the scenario where a partition table uses PBE, dynamic partition pruning may possibly cause core dump. To solve this problem, set **enable_resource_track** to **off**. This parameter has been incorporated into Recommended Parameter Settings. \ No newline at end of file +In the scenario where a partition table is encrypted by PBE (password-based encryption), dynamic partition pruning may possibly cause a core dump. To solve this problem, set **enable_resource_track** to **off**. This parameter has been incorporated into Recommended Parameter Settings. \ No newline at end of file -- Gitee From ea666513400a06e1ccdfc58629ab1979e1d22b22 Mon Sep 17 00:00:00 2001 From: zcp100_zcp100 Date: Mon, 4 Jul 2022 09:19:29 +0800 Subject: [PATCH 3/4] =?UTF-8?q?update(mogdb):=E4=BF=AE=E6=94=B9tococ=5Fabo?= =?UTF-8?q?ut?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- product/zh/docs-mogdb/v3.0/toc_about.md | 1 - 1 file changed, 1 deletion(-) diff --git a/product/zh/docs-mogdb/v3.0/toc_about.md b/product/zh/docs-mogdb/v3.0/toc_about.md index 5e32d3e6..0a78ef87 100644 --- a/product/zh/docs-mogdb/v3.0/toc_about.md +++ b/product/zh/docs-mogdb/v3.0/toc_about.md @@ -15,6 +15,5 @@ + [wal2json](/about-mogdb/open-source-components/5-wal2json-extention-for-mogdb&opengauss.md) + [mog_filedump](/about-mogdb/open-source-components/mog_filedump.md) + [mog_xlogdump](/about-mogdb/open-source-components/mog_xlogdump.md) - + [PostGIS](/about-mogdb/open-source-components/PostGIS.md) + [使用限制](/about-mogdb/usage-limitations.md) + [法律声明](/about-mogdb/terms-of-use.md) -- Gitee From 3846bc5468526e086489d622ba7ad7cb7b859617 Mon Sep 17 00:00:00 2001 From: zcp100_zcp100 Date: Wed, 6 Jul 2022 15:14:11 +0800 Subject: [PATCH 4/4] =?UTF-8?q?update(mogdb):3.0=E6=96=B0=E7=89=B9?= =?UTF-8?q?=E6=80=A7=E4=BF=AE=E6=94=B9?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../v3.0/about-mogdb/mogdb-new-feature/3.0.0.md | 13 ++----------- 1 file changed, 2 insertions(+), 11 deletions(-) diff --git a/product/zh/docs-mogdb/v3.0/about-mogdb/mogdb-new-feature/3.0.0.md b/product/zh/docs-mogdb/v3.0/about-mogdb/mogdb-new-feature/3.0.0.md index f1cdfa09..122cb7e9 100644 --- a/product/zh/docs-mogdb/v3.0/about-mogdb/mogdb-new-feature/3.0.0.md +++ b/product/zh/docs-mogdb/v3.0/about-mogdb/mogdb-new-feature/3.0.0.md @@ -51,7 +51,7 @@ MogDB 3.0.0版本于2022年6月30日发布。3.0.0版本基于2.1版本进一步 ### 2.5 Copy导入优化 -- 利用CPU SSE4.2指令集，在COPY导入阶段利用SIMD特性提升COPY导入的性能。 +- 利用CPU的指令集，在COPY数据解析阶段进行并行操作，进而提升COPY导入性能。该特性目前仅限x86 CPU架构。 **相关页面**：[COPY导入优化](../../characteristic-description/enterprise-level-features/20-copy-import-optimization.md) @@ -93,7 +93,6 @@ PostGis是一个空间数据库系统，能够提供空间对象、空间索引 **函数** - INSTRB -- NEW_TIME - NLS_CHARSET_ID - NLS_CHARSET_NAME - NLS_LOWER @@ -103,14 +102,10 @@ PostGis是一个空间数据库系统，能够提供空间对象、空间索引 - REPLACE - SHOW - SHOW_PARAMETER -- SYS_EXTRACT_UTC -- TO_DSINTERVAL - TO_TIMESTAMP -- TO_TIMESTAMP_TZ - TO_YMINTERVAL - TZ_OFFSET - NULLIF -- VSIZE - Ratio_to_report **Package** @@ -191,8 +186,4 @@ PostGis是一个空间数据库系统，能够提供空间对象、空间索引 | openEuler 20.03LTS | ARM（鲲鹏）、X86_64（Intel，AMD，海光，兆芯） | | 银河麒麟V10 | ARM（鲲鹏）、X86_64（Intel，AMD，海光，兆芯） | | 统信UOS V20-D / V20-E | ARM（鲲鹏）、X86_64（Intel，AMD，海光，兆芯） | -| 统信UOS V20-A | X86_64（Intel，AMD，海光，兆芯） | - -## 5. 已知问题 - -- 在分区表使用PBE场景下，动态分区裁剪小概率可能会导致core dump，规避方案为暂时禁用resource track，也即设置enable_resource_track=off。该参数已经纳入推荐参数设置。 \ No newline at end of file +| 统信UOS V20-A | X86_64（Intel，AMD，海光，兆芯） | \ No newline at end of file -- Gitee